-- Logs begin at Tue 2022-09-27 03:40:37 UTC, end at Tue 2022-09-27 04:41:31 UTC. -- Sep 27 04:11:25.969461 dv-cinder-859186-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 27 04:11:25.977419 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 04:11:25 2022] *** Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: nodename: dv-cinder-859186-3-iscsi Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: machine: x86_64 Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: clock source: unix Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: pcre jit disabled Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: detected number of CPU cores: 4 Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: current working directory: / Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: detected binary path: /usr/bin/uwsgi-core Sep 27 04:11:25.979924 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: your processes number limit is 47680 Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: your memory page size is 4096 bytes Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: detected max file descriptor number: 2048 Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: lock engine: pthread robust mutexes Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: thunder lock: enabled Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 27 04:11:25.980556 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 04:11:26.024620 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: Python main interpreter initialized at 0x55de1108c9c0 Sep 27 04:11:26.024620 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: python threads support enabled Sep 27 04:11:26.024620 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: your server socket listen backlog is limited to 100 connections Sep 27 04:11:26.024620 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: your mercy for graceful operations on workers is 90 seconds Sep 27 04:11:26.025026 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 04:11:26.025026 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: *** Operational MODE: preforking *** Sep 27 04:11:26.025298 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: *** uWSGI is running in multiple interpreter mode *** Sep 27 04:11:26.025298 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: spawned uWSGI master process (pid: 105427) Sep 27 04:11:26.025555 dv-cinder-859186-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 27 04:11:26.027946 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: spawned uWSGI worker 1 (pid: 105428, cores: 1) Sep 27 04:11:26.027946 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: spawned uWSGI worker 2 (pid: 105429, cores: 1) Sep 27 04:11:26.027946 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 04:11:26.677788 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 04:11:26.678349 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 04:11:27.747991 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:11:27.747991 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: from cryptography.utils import int_from_bytes Sep 27 04:11:27.748705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:11:27.748705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: from cryptography.utils import int_from_bytes Sep 27 04:11:27.749623 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:11:27.749623 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: from cryptography.utils import int_from_bytes Sep 27 04:11:27.750090 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:11:27.750090 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: from cryptography.utils import int_from_bytes Sep 27 04:11:30.206943 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 04:11:30.207522 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 04:11:30.207872 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 04:11:30.207967 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 04:11:30.208057 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 04:11:30.208138 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 04:11:30.208230 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 04:11:30.208311 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 04:11:30.208390 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 04:11:30.208476 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 04:11:30.208646 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.208789 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.209010 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.209105 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.209265 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.209957 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210121 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210275 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210444 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210598 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210749 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.210905 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211060 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211219 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211369 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211689 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211851 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.211949 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212107 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212238 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212414 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212555 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-859186-3-iscsi {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212730 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-859186-3-iscsi {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.212867 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213046 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213165 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213347 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213469 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213653 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213791 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.213985 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214082 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214290 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214390 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214603 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214697 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.214902 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215145 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215400 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215576 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215691 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215878 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.215998 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216183 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216303 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216510 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216624 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216817 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.216954 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217119 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217217 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217435 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217520 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217737 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.217841 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218077 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218217 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218380 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218473 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218764 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-859186-3-iscsi {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.218849 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-859186-3-iscsi {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219074 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219168 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219377 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219470 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219682 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.219773 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220082 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220182 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220385 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220470 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220686 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220770 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.220995 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221305 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221398 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221609 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221774 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.221956 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222049 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222267 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222359 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222577 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222661 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222886 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.222978 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223178 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223270 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223478 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223563 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223777 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.223871 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224082 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224217 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224406 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224514 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224673 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224804 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.224903 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.225581 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.225762 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.225872 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226089 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226192 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226403 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226490 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226714 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226812 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.226961 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227048 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227205 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227292 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227514 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227601 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227823 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.227910 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228122 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228216 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228423 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228524 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228722 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.228818 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229019 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229114 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229317 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229411 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229613 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.229937 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230034 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230236 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230321 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230546 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230631 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230850 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.230944 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231160 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231252 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231456 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231547 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231764 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.231851 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232145 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232230 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232390 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232476 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232688 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.232774 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233016 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233110 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233323 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233418 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233620 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233727 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.233955 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234052 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234266 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234370 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234564 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234660 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234858 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.234944 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235154 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235325 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235463 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235549 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235783 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.235874 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236078 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236174 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236321 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236407 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236614 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236706 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.236911 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237009 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237218 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237307 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237518 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237604 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237839 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.237943 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238138 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238231 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238429 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238515 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238724 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.238821 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239108 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239317 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239410 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239613 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239908 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.239994 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240205 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240291 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240499 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240616 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240833 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.240920 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241131 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241223 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241427 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241514 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.241842 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242067 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242153 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242364 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242450 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242661 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242764 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.242956 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243042 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243255 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243341 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243562 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243657 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243859 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.243970 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244155 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244244 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244464 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244568 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244797 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.244878 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245042 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245131 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245348 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245440 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245641 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245734 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.245963 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246053 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246272 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246359 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246564 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246673 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246861 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.246948 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247164 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247251 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247467 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247563 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247763 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.247857 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248057 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248143 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248350 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248444 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248642 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248728 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.248937 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249023 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249229 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249327 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249522 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249614 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249833 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.249923 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250140 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250228 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250449 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250536 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250749 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.250833 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251045 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251131 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251342 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251427 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251647 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251739 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:11:30.251972 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252059 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252293 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252380 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252626 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252718 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.252930 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253016 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253253 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253339 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253573 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253666 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253908 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.253995 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254212 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254299 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254520 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254617 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254837 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.254933 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255139 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255224 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255440 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255541 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255742 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.255834 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256048 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256152 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256391 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256494 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256714 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.256801 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257025 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257127 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257337 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257488 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257645 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257790 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.257975 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258063 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258287 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258389 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258602 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258693 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.258914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259008 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259219 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259316 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259551 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259636 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259886 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.259973 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260193 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260280 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260509 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260603 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260819 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.260912 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.261377 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.261482 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.261680 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.261828 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262010 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262097 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262344 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262434 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262646 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.262951 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263049 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263264 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263350 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263569 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263656 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263884 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.263982 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.264191 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.264312 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.264494 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.264622 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.264799 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265014 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265105 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265185 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265394 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265721 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.265865 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266051 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266137 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266366 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266452 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266672 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266759 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.266975 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267068 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267378 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267590 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267678 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267893 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.267979 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268196 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268282 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268493 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268580 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268790 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.268875 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269087 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269172 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269389 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269490 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269683 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.269788 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270020 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270109 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270340 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270427 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270639 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270731 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.270962 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.271057 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.271260 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.271345 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.271563 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.271650 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.272385 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.272497 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.272691 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.272778 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.272991 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273084 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273295 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273387 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273599 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273685 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.273920 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274237 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274331 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274539 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274638 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274837 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.274933 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275149 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275245 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275434 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275529 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.275817 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276037 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276129 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276341 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276427 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276644 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276737 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.276950 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277043 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277252 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277337 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277563 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277657 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277889 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.277978 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278193 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278282 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278517 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278619 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278839 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.278934 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279137 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279223 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279453 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279547 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279766 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.279862 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280075 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280167 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280370 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280464 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280670 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280755 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.280968 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281265 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281358 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281561 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281653 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281887 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.281976 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282185 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282272 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282481 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282566 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282776 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.282861 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283069 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283162 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283362 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283456 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283673 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283765 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.283972 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284059 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284266 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284351 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284563 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284650 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284860 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.284947 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285193 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285291 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285489 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285584 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285840 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.285930 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286169 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286265 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286478 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286577 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286791 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.286876 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.287094 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.287181 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.287399 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288081 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288256 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288343 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288653 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288864 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.288958 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289169 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289256 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289482 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289577 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289808 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.289909 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290125 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290211 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290429 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290516 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290742 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.290835 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291043 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291129 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291353 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291439 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291650 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291743 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.291957 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292049 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292285 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292382 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292607 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292701 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.292999 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293227 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293320 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293529 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293615 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293861 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.293956 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294165 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294260 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294468 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294609 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294752 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.294907 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295049 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295198 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295370 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295456 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295648 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295735 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.295957 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296056 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296260 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296353 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296572 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296658 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296872 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.296966 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297179 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297273 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297476 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297569 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297802 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.297908 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298124 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298210 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298447 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298533 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298750 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.298837 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299055 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299140 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299369 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299455 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299672 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299758 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.299972 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300058 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300274 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300360 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300570 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300657 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300867 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.300953 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301168 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301263 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301471 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301556 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301793 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.301885 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302097 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302201 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302393 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302481 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302700 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.302785 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303002 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303120 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303325 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303420 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303624 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303717 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.303930 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304028 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304243 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304336 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304547 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304633 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304863 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.304951 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305173 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305261 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305469 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305784 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.305885 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306104 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306193 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306407 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306493 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306703 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.306790 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307000 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307091 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307298 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307383 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307660 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307755 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.307930 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308016 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308229 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308323 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308538 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308624 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308849 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.308934 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309152 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309258 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309460 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309556 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309787 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.309889 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310087 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310184 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310398 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310491 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310703 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.310795 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311003 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311101 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311306 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311393 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311608 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311695 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.311910 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312007 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312211 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312296 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312524 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312616 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312824 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.312918 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313127 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313212 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313428 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313515 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313725 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.313876 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.314049 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.314135 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.314358 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.314452 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.314914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315002 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315240 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315343 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315552 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315658 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315861 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.315953 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316195 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316289 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316505 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316603 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316827 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.316920 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317144 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317238 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317449 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317535 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317786 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.317876 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.318075 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.318163 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.318372 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.318457 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319178 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319283 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319498 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319583 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319797 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.319883 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320097 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320195 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320392 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320484 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320693 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320779 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.320988 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321081 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321284 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321369 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321578 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321662 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321897 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.321997 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322205 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322300 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322502 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322588 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322805 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.322900 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323101 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323187 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323414 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323501 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323713 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.323800 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324010 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324095 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324305 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324390 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324602 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.324931 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325017 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325228 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325314 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325571 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325706 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325897 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.325985 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326228 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326327 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326527 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326622 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326821 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.326907 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327135 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327230 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327429 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327514 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327726 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.327812 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328119 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328321 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328415 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328621 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328706 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.328914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329008 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329215 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329301 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329515 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329600 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329839 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.329929 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330142 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330229 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330436 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330522 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330735 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.330829 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331029 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331126 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331335 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331420 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331634 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.331928 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332023 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332222 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332309 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332521 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332608 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332821 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.332906 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333113 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333206 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333414 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333500 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333706 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.333821 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.334036 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.334131 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.334379 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.334961 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335168 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335265 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335468 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335574 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335773 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.335862 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336073 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336172 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336368 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336467 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336681 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336768 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.336982 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337068 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337278 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337365 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337573 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337658 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337897 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.337986 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338193 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338282 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338573 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338791 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.338886 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339101 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339189 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339408 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339494 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339704 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.339791 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340006 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340093 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340303 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340389 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340603 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340896 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.340986 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341194 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341280 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341487 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341581 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341805 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.341909 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342104 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342204 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342398 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342492 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342691 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.342777 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343001 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343087 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343300 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343395 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343596 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343682 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343893 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.343980 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344187 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344274 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344517 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344603 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344812 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.344900 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345158 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345245 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345456 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345543 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345794 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.345886 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346093 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346190 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346397 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346490 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346699 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.346785 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347004 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347099 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347301 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347394 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347608 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347693 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.347910 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348044 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348213 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348307 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348508 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348602 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348803 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.348889 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349113 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349207 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349418 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349516 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349719 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.349826 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350077 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350184 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350389 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350483 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350703 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350797 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.350997 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351083 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351298 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351391 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351600 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351687 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351900 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.351986 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352201 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352288 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352501 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352585 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352804 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.352888 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353106 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353192 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353409 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353501 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353710 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.353832 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354036 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354131 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354335 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354423 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354637 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354722 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.354939 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355025 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355236 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355320 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355548 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355642 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355850 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.355983 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356146 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356231 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356448 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356546 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356745 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.356830 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357047 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357131 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357343 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357437 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357792 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.357883 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358075 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358171 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358379 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358473 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358681 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358774 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.358984 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359075 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359287 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359374 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359596 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359688 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359903 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.359989 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.360199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.360285 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.360604 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.360697 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.360912 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361007 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361215 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361303 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361517 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361602 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361840 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.361937 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362139 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362228 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362444 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362530 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.362854 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363057 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363150 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363353 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363438 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363648 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363746 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.363961 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364053 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364257 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364342 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364569 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364663 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.364973 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.365059 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.365277 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.365409 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.365597 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.365757 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366033 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366121 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366337 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366422 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366642 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366750 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.366956 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367056 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367269 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367355 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367585 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367672 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367884 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.367971 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368190 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368276 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368496 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368582 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368813 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.368900 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369128 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369214 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369432 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369530 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369778 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.369870 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370065 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370152 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370379 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370465 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370692 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370779 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.370998 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.371088 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.371310 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.371405 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.371711 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.371798 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372026 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372118 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372330 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372645 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372734 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.372961 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373047 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373263 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373358 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373561 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373647 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373888 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.373977 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374190 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374292 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374491 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374579 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374802 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.374888 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375100 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375187 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375405 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375498 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375709 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.375844 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376012 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376106 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376311 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376406 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376608 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376695 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.376921 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377010 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377219 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377305 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377532 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377619 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377861 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.377950 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378177 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378263 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378495 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378603 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378796 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.378884 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379109 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379196 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379413 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379507 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379716 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.379803 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380098 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380313 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380406 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380644 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380731 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.380992 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.381120 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.381793 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.381896 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382108 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382196 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382428 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382521 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382733 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.382820 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383059 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383145 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383378 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383464 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383691 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383786 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.383996 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384082 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384307 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384393 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384627 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384714 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.384924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385017 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385221 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385314 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385525 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385612 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385846 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.385946 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.386401 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.386501 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.386700 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.386792 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387001 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387094 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387297 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387392 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387595 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387681 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387894 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.387980 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388207 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388301 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388506 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388591 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388820 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.388906 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389123 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389218 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389420 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389506 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389718 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.389860 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390040 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390141 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390372 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390458 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390669 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390761 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.390989 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391076 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391292 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391401 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391607 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391693 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.391917 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392004 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392220 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392326 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392530 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392630 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392838 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.392923 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393145 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393231 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393452 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393546 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393778 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.393870 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394089 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394186 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394397 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394509 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.394798 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.395016 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.395103 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.395525 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.395619 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.395849 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396011 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396158 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396244 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396464 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396559 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396769 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.396896 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.397639 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.397725 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.397976 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398073 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398287 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398374 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398596 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398682 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398900 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.398987 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399204 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399289 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399524 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399616 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399839 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.399928 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400157 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400244 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400484 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400578 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400794 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.400887 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401104 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401228 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401476 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401606 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401724 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.401898 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402080 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402198 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402390 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402505 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402694 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.402806 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403002 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403122 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403311 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403430 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403617 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403730 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.403921 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404042 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404225 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404344 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404532 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404644 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404838 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.404960 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.405143 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.405256 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.405447 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.405560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406232 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406320 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406320 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406466 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406589 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406752 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406838 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406918 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.406998 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407078 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407293 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407390 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407602 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407690 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407910 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.407997 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.408423 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.408518 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.408733 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.408820 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409060 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409145 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409376 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409470 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409691 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.409808 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410028 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410115 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410339 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410428 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410659 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410744 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.410956 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411042 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411255 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411350 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411558 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411645 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411861 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.411948 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412157 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412243 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412497 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412587 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412803 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.412890 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413104 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413191 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413415 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413508 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.413885 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.414265 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.414354 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.414604 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.414694 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.414914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.415011 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.415234 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.415320 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.415550 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.415671 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418150 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418312 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418312 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418525 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418643 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418751 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418858 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.418973 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419082 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419188 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419293 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419400 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419519 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419636 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419742 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419847 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.419951 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420063 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420183 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420289 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420359 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420434 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420434 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420594 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420687 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420769 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420848 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420927 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.420927 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421069 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421157 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421245 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421332 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421413 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421500 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421589 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421674 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421858 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.421946 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422026 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422106 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422199 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422281 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422361 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422441 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422520 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422599 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422690 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422856 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.422953 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423159 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423246 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423452 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423548 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423754 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.423906 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424101 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424187 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424405 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424493 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424703 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.424790 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425016 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425115 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425313 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425406 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425608 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425695 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.425944 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426044 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426337 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426561 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426646 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426864 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.426967 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427165 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427259 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427500 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427585 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427808 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.427904 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.428359 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.428495 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.428721 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.428819 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.428988 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429091 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429313 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429408 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429637 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429732 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.429967 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430064 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430269 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430362 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430567 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430653 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430885 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.430990 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431190 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431276 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431495 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431583 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431799 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.431932 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432101 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432186 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432402 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432490 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432704 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.432798 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433006 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433094 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433307 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433401 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433608 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433695 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.433937 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434038 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434240 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434336 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434541 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434626 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434842 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.434928 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435143 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435314 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435447 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435535 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435776 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.435862 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436096 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436231 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436401 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436521 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436707 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.436821 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437018 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437126 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437317 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437404 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437618 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437703 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.437944 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438065 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438246 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438373 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438553 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438684 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.438876 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.439007 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.439185 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.439316 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.439487 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.439618 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440154 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440293 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440467 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440608 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440784 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.440926 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441101 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441244 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441410 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441552 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441728 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.441902 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442068 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442210 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442373 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442517 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442690 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442832 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.442996 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.443138 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.443318 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.443459 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.443651 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.444311 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.444493 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.444635 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.444812 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.444954 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445113 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445257 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445413 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445558 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445714 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.445881 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446047 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446186 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446361 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446530 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446735 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.446901 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447048 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447182 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447269 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447359 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447525 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447666 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447822 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.447966 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448122 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448266 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448420 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448565 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448718 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.448863 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449166 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449317 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449464 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449620 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449788 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.449942 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450080 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450240 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450384 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450538 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450684 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450838 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.450982 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451142 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451439 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451584 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451738 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.451884 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452038 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452186 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452336 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452482 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452632 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452779 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.452956 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453103 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453280 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453427 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453593 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453813 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.453953 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454104 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454260 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454410 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454576 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454725 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.454881 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455031 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455196 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455344 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455511 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455655 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455821 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.455966 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456125 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456270 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456454 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456592 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456763 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.456896 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457114 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457233 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457383 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457486 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457689 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.457819 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458109 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458325 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458421 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458644 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458735 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.458964 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.459057 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.459268 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.459400 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460150 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460244 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460474 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460569 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460791 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.460879 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461102 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461195 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461407 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461498 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461767 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.461880 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462141 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462362 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462458 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462663 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462749 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.462969 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463054 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463273 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463366 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463583 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463675 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463886 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.463984 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464191 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464277 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464501 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464587 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464812 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.464976 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465129 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465214 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465447 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465532 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465787 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.465890 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466091 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466176 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466407 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466492 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466748 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.466851 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467052 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467138 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467387 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467473 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467710 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.467805 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468018 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468104 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468324 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468409 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468626 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468712 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:11:30.468864 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 04:11:30.469009 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 04:11:30.620323 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO dbcounter [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Registered counter for database nova_api Sep 27 04:11:30.620644 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO dbcounter [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Registered counter for database nova_api Sep 27 04:11:30.641365 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_db.sqlalchemy.engines [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.642846 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_db.sqlalchemy.engines [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.648727 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writer thread running {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.649878 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writer thread running {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.705678 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.context [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),68475614-ed24-4318-8ce9-852b74c0d16a(cell1) {{(pid=105429) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 04:11:30.705913 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.context [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),68475614-ed24-4318-8ce9-852b74c0d16a(cell1) {{(pid=105428) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 04:11:30.709586 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:11:30.709794 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:11:30.710034 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:11:30.710196 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:11:30.711570 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:11:30.711764 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:11:30.712283 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:11:30.712494 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:11:30.712719 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:11:30.712941 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:11:30.714732 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_concurrency.lockutils [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105429) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:11:30.714968 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:11:30.716711 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO dbcounter [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Registered counter for database nova_cell0 Sep 27 04:11:30.716963 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO dbcounter [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Registered counter for database nova_cell0 Sep 27 04:11:30.720681 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO dbcounter [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Registered counter for database nova_cell1 Sep 27 04:11:30.721157 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO dbcounter [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Registered counter for database nova_cell1 Sep 27 04:11:30.727983 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_db.sqlalchemy.engines [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.729237 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_db.sqlalchemy.engines [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.731440 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_db.sqlalchemy.engines [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.733110 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_db.sqlalchemy.engines [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.735760 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writer thread running {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.738005 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writer thread running {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.738993 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writer thread running {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.741769 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writer thread running {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.882318 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO dbcounter [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Registered counter for database nova_cell0 Sep 27 04:11:30.883180 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO dbcounter [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Registered counter for database nova_cell0 Sep 27 04:11:30.891367 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG oslo_db.sqlalchemy.engines [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.894914 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_db.sqlalchemy.engines [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:11:30.896215 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writer thread running {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:30.900613 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writer thread running {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:11:31.231283 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: WARNING keystonemiddleware.auth_token [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:11:31.236581 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: WARNING keystonemiddleware.auth_token [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:11:31.238541 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: WARNING keystonemiddleware.auth_token [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:11:31.243940 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: WARNING keystonemiddleware.auth_token [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:11:31.576234 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: WARNING keystonemiddleware.auth_token [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:11:31.582119 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: WARNING keystonemiddleware.auth_token [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:11:31.585714 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: WARNING keystonemiddleware.auth_token [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:11:31.587347 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55de1108c9c0 pid: 105428 (default app) Sep 27 04:11:31.589386 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:11:31.590612 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] 10.222.0.26 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001768 Sep 27 04:11:31.591018 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 788 bytes} [Tue Sep 27 04:11:31 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:11:31.591674 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: WARNING keystonemiddleware.auth_token [None req-a0d72e80-b952-477e-b6e1-6add4e1f2d5c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:11:31.597009 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55de1108c9c0 pid: 105429 (default app) Sep 27 04:12:52.550844 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:12:52.551849 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Sep 27 04:12:52.552248 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:12:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:12:52.569203 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_api:SELECT=3 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:52.570339 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_cell1:SELECT=3 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:52.571375 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_cell0:SELECT=3 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:52.572117 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:53.443556 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-a9228b27-e92e-45c5-a628-c5af6791e2cc admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:12:53.476301 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-a9228b27-e92e-45c5-a628-c5af6791e2cc admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.917150 Sep 27 04:12:53.477030 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 1/3] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 04:12:52 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:12:56.083971 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:12:56.089372 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-33f56350-3aff-40a3-80ae-aabdb5eebd8d None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.005762 Sep 27 04:12:56.090183 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 3/4] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:12:56 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:12:56.534607 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ea7adf7b-c423-4796-b8b5-3ba9b6d9e3cd admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 256, "disk": 1, "os-flavor-access:is_public": true, "name": "cirros256", "rxtx_factor": 1.0, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:12:56.580726 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ea7adf7b-c423-4796-b8b5-3ba9b6d9e3cd admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.483692 Sep 27 04:12:56.581293 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 2/5] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:12:56 2022] POST /compute/v2.1/flavors => generated 407 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:12:56.595016 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_api:SELECT=3 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:56.597029 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_cell1:SELECT=3 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:56.597798 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_cell0:SELECT=3 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:56.598732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:12:56.808395 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-24e62b7d-df78-43d6-8ba7-b13b2fd55033 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:12:56.878326 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-24e62b7d-df78-43d6-8ba7-b13b2fd55033 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.288357 Sep 27 04:12:56.878922 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 4/6] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 04:12:56 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:12:59.538996 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ea7adf7b-c423-4796-b8b5-3ba9b6d9e3cd admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:12:59.540020 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ea7adf7b-c423-4796-b8b5-3ba9b6d9e3cd admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Sep 27 04:12:59.540414 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 3/7] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:12:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:00.191851 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-6c4059a1-57ab-45af-9aba-691da734ab79 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "id": "d1", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds512M", "rxtx_factor": 1.0, "swap": 0, "disk": 5, "vcpus": 1}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:00.232566 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-6c4059a1-57ab-45af-9aba-691da734ab79 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.685575 Sep 27 04:13:00.237801 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 5/8] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:12:59 2022] POST /compute/v2.1/flavors => generated 404 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:00.246939 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-d17a2cf7-373f-45cb-acfd-b526781c3fa5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:00.296141 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-d17a2cf7-373f-45cb-acfd-b526781c3fa5 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.053705 Sep 27 04:13:00.296956 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 4/9] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 04:13:00 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:02.902023 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-6c4059a1-57ab-45af-9aba-691da734ab79 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:02.903047 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-6c4059a1-57ab-45af-9aba-691da734ab79 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 Sep 27 04:13:02.903474 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:03.315160 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-745e6dcd-b698-4f73-a106-f7f80bed7863 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 1024, "name": "ds1G", "swap": 0, "id": "d2", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 10}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:03.334660 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-745e6dcd-b698-4f73-a106-f7f80bed7863 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.424394 Sep 27 04:13:03.335288 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 5/11] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:02 2022] POST /compute/v2.1/flavors => generated 404 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:03.347999 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-9c10d23d-2aab-4a99-a297-0b80936babe6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:03.377031 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-9c10d23d-2aab-4a99-a297-0b80936babe6 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032910 Sep 27 04:13:03.377661 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 7/12] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 04:13:03 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:05.986990 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-745e6dcd-b698-4f73-a106-f7f80bed7863 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:05.988009 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-745e6dcd-b698-4f73-a106-f7f80bed7863 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 Sep 27 04:13:05.988407 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 6/13] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:06.394039 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-156449c4-7838-46ce-9e0e-051425bebe30 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "name": "ds2G", "disk": 10, "rxtx_factor": 1.0, "id": "d3", "os-flavor-access:is_public": true, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:06.423274 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-156449c4-7838-46ce-9e0e-051425bebe30 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.428126 Sep 27 04:13:06.424089 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 8/14] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:05 2022] POST /compute/v2.1/flavors => generated 404 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:06.436947 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-5caa0162-8abb-4df9-9271-59fe8bcde610 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:06.465478 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-5caa0162-8abb-4df9-9271-59fe8bcde610 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032346 Sep 27 04:13:06.469771 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 7/15] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 04:13:06 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:13:09.095830 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-156449c4-7838-46ce-9e0e-051425bebe30 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:09.097346 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-156449c4-7838-46ce-9e0e-051425bebe30 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002409 Sep 27 04:13:09.098366 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 9/16] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:09 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:09.522057 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 4096, "disk": 20, "id": "d4", "os-flavor-access:is_public": true, "name": "ds4G", "vcpus": 4, "swap": 0}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:09.541225 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.435656 Sep 27 04:13:09.541857 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 8/17] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:09 2022] POST /compute/v2.1/flavors => generated 404 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:09.555891 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-a9ed8dbe-774f-489a-81b6-212ee4950e7a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:09.584750 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-a9ed8dbe-774f-489a-81b6-212ee4950e7a admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033963 Sep 27 04:13:09.585433 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 10/18] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 04:13:09 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:12.702905 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:12.703854 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 27 04:13:12.704256 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 9/19] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:13.129488 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-d509edec-0bb6-4212-9286-141e9e066fd7 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:13.158019 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-d509edec-0bb6-4212-9286-141e9e066fd7 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.447797 Sep 27 04:13:13.158601 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 11/20] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 04:13:12 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:15.791786 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:15.792738 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-fa336a3b-fe22-4714-aa7a-aa4987e61195 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Sep 27 04:13:15.793151 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 10/21] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:16.196407 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-ec9ee590-591c-49f4-8408-a2ee7729ef6e admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "name": "m1.tiny", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "id": "1", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 512}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:16.216487 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-ec9ee590-591c-49f4-8408-a2ee7729ef6e admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.417220 Sep 27 04:13:16.217027 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 12/22] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:15 2022] POST /compute/v2.1/flavors => generated 402 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:16.229444 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-5f631af6-3eb8-4a48-80a7-ddcf1999d929 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:16.256402 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-5f631af6-3eb8-4a48-80a7-ddcf1999d929 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030523 Sep 27 04:13:16.257170 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 11/23] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 04:13:16 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:18.839739 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-ec9ee590-591c-49f4-8408-a2ee7729ef6e admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:18.840753 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-ec9ee590-591c-49f4-8408-a2ee7729ef6e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Sep 27 04:13:18.841155 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:19.223865 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b945722-a626-424c-b341-f928a62ae583 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "id": "2", "name": "m1.small", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 2048, "vcpus": 1, "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:19.245917 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b945722-a626-424c-b341-f928a62ae583 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.398589 Sep 27 04:13:19.246475 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 12/25] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:19.258352 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-8d3c61c4-a890-46b5-b9ef-347529843060 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:19.281117 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-8d3c61c4-a890-46b5-b9ef-347529843060 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026426 Sep 27 04:13:19.281680 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 14/26] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 04:13:19 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:21.902112 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b945722-a626-424c-b341-f928a62ae583 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:21.903794 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b945722-a626-424c-b341-f928a62ae583 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002145 Sep 27 04:13:21.904206 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:21 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:22.354224 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f0502203-324e-4fad-84d7-36cf26eb767c admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "name": "m1.medium", "disk": 40, "id": "3"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:22.379470 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f0502203-324e-4fad-84d7-36cf26eb767c admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.469010 Sep 27 04:13:22.381809 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 15/28] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:21 2022] POST /compute/v2.1/flavors => generated 406 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:22.392584 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-a641c9b1-4d93-47e6-a502-d2a0404ee21f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:22.429052 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-a641c9b1-4d93-47e6-a502-d2a0404ee21f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040188 Sep 27 04:13:22.430251 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 14/29] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 04:13:22 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:25.045782 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f0502203-324e-4fad-84d7-36cf26eb767c admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:25.046808 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f0502203-324e-4fad-84d7-36cf26eb767c admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001484 Sep 27 04:13:25.047206 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 16/30] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:25.577681 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-2d969a6f-fa84-4194-ab59-88da70209ddb admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "rxtx_factor": 1.0, "id": "4", "name": "m1.large", "ram": 8192, "swap": 0, "vcpus": 4, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:25.583340 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:13:25.598895 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-2d969a6f-fa84-4194-ab59-88da70209ddb admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.545462 Sep 27 04:13:25.601793 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 15/31] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:25.611467 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-44e5e2a6-58cc-4710-96a6-ef1be6d578a7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:25.637898 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-44e5e2a6-58cc-4710-96a6-ef1be6d578a7 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030121 Sep 27 04:13:25.638816 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 17/32] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 04:13:25 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:13:28.267649 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-2d969a6f-fa84-4194-ab59-88da70209ddb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:28.268847 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-2d969a6f-fa84-4194-ab59-88da70209ddb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001645 Sep 27 04:13:28.269377 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 16/33] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:28.711915 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "rxtx_factor": 1.0, "ram": 16384, "id": "5", "swap": 0, "name": "m1.xlarge", "disk": 160, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:28.719323 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:13:28.737697 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.461968 Sep 27 04:13:28.738297 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 18/34] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 04:13:28 2022] POST /compute/v2.1/flavors => generated 408 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:13:28.750244 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-d113bb67-a1db-4d08-927d-5aaac52e4761 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:28.776592 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-d113bb67-a1db-4d08-927d-5aaac52e4761 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029989 Sep 27 04:13:28.777174 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 17/35] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 04:13:28 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:13:46.176726 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:46.177687 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Sep 27 04:13:46.178705 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:13:46 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:46.189333 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:13:46.561113 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-5deffd80-3a28-43e5-87e6-5f63f03e5dc9 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:46.578958 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-5deffd80-3a28-43e5-87e6-5f63f03e5dc9 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.394152 Sep 27 04:13:46.579492 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 18/37] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 04:13:46 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:13:47.880629 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:47.882207 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f7a92422-5439-4887-956e-be8b99568706 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001929 Sep 27 04:13:47.882607 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 20/38] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:48.398078 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-2ff005ce-aea6-43bd-a7a2-ac8e662ef37a admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:48.403473 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.wsgi [None req-2ff005ce-aea6-43bd-a7a2-ac8e662ef37a admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-2ff005ce-aea6-43bd-a7a2-ac8e662ef37a admin admin] Request method failure captured: Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept: application/json Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept-Encoding: gzip, deflate Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Connection: keep-alive Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Content-Length: 0 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Host: 10.222.0.26 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-System-Scope: None Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: User-Agent: python-novaclient Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Auth-Token: gAAAAABjMnh7I-zQ7bzyL0vgRnmO0SD0aD0P7fvri9vH-RSx1DXp8TfsxYSoBKydSbstrTjiD9zxNZvGQ4bW3tb7u5Qn-3cw0OKlVyCEhJWQoYOTFBGXUjnE70rSX-geQek_7OU8pQErcMvpI9uTnQu6LUqvEAf-0M8MED7jOattdX65ZuKk0yM Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Id: None Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Name: None Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Identity-Status: Confirmed Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Is-Admin-Project: True Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Id: default Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Name: Default Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Name: admin Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Role: admin,reader,member Sep 27 04:13:48.406814 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Roles: admin,reader,member Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant: admin Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Name: admin Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User: admin Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Id: default Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Name: Default Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Name: admin Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: method: > Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: exception: Flavor 42 could not be found. Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: action_args: {'id': '42'} Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:48.408431 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:48.409557 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.411013 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-2ff005ce-aea6-43bd-a7a2-ac8e662ef37a admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105429) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:48.412816 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-2ff005ce-aea6-43bd-a7a2-ac8e662ef37a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.388991 Sep 27 04:13:48.412816 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 04:13:48 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 390 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:48.418554 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-47fb7abf-8379-4e10-a346-38a95687a28d admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:48.441044 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-47fb7abf-8379-4e10-a346-38a95687a28d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026054 Sep 27 04:13:48.441583 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:48 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:48.451984 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-15f7765f-dbaf-46ac-bdd2-5b465436c816 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:48.473452 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-15f7765f-dbaf-46ac-bdd2-5b465436c816 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.025928 Sep 27 04:13:48.474010 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:48 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:48.483785 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-c4b5351d-0b15-4f8c-85db-e40fcd76c6af admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:48.489039 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.wsgi [None req-c4b5351d-0b15-4f8c-85db-e40fcd76c6af admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-c4b5351d-0b15-4f8c-85db-e40fcd76c6af admin admin] Request method failure captured: Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept: application/json Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept-Encoding: gzip, deflate Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Connection: keep-alive Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Content-Length: 0 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Host: 10.222.0.26 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-System-Scope: None Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: User-Agent: python-novaclient Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Auth-Token: gAAAAABjMnh7I-zQ7bzyL0vgRnmO0SD0aD0P7fvri9vH-RSx1DXp8TfsxYSoBKydSbstrTjiD9zxNZvGQ4bW3tb7u5Qn-3cw0OKlVyCEhJWQoYOTFBGXUjnE70rSX-geQek_7OU8pQErcMvpI9uTnQu6LUqvEAf-0M8MED7jOattdX65ZuKk0yM Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Id: None Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Name: None Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Identity-Status: Confirmed Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Is-Admin-Project: True Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Id: default Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Name: Default Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Name: admin Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Role: admin,reader,member Sep 27 04:13:48.492925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Roles: admin,reader,member Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant: admin Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Name: admin Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User: admin Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Id: default Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Name: Default Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Name: admin Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: method: > Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: exception: Flavor 42 could not be found. Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: action_args: {'id': '42'} Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:48.494491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:48.495425 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:48.496760 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-c4b5351d-0b15-4f8c-85db-e40fcd76c6af admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105428) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:48.498020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-c4b5351d-0b15-4f8c-85db-e40fcd76c6af admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014155 Sep 27 04:13:48.498020 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 22/42] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 04:13:48 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:49.709072 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-15f7765f-dbaf-46ac-bdd2-5b465436c816 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:49.710119 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-15f7765f-dbaf-46ac-bdd2-5b465436c816 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Sep 27 04:13:49.710513 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 21/43] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:50.221946 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-d6c34060-23da-410c-bb69-bf08dfafe80a admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:50.227612 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.wsgi [None req-d6c34060-23da-410c-bb69-bf08dfafe80a admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-d6c34060-23da-410c-bb69-bf08dfafe80a admin admin] Request method failure captured: Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept: application/json Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept-Encoding: gzip, deflate Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Connection: keep-alive Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Content-Length: 0 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Host: 10.222.0.26 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-System-Scope: None Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: User-Agent: python-novaclient Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Auth-Token: gAAAAABjMnh9JyMhSTKtECrbxg9Pfl6ILz1dPTLpg_Yun4TvC4czgdiRIqC7JGOZyZekQ-CaMDn39feQgY9GYn-jlL-DbXke7UnnkL0Fu1h12dFecvT3M7iPStqNOZbe_otIqzIrW_7sPys3Rgap38Yfg1zFs5EgWvdVXfNP5jRDfEtTbmrB28U Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Id: None Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Name: None Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Identity-Status: Confirmed Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Is-Admin-Project: True Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Id: default Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Name: Default Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Name: admin Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Role: admin,reader,member Sep 27 04:13:50.229215 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Roles: admin,reader,member Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant: admin Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Name: admin Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User: admin Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Id: default Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Name: Default Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Name: admin Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: method: > Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: exception: Flavor 84 could not be found. Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: action_args: {'id': '84'} Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:50.230924 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:50.232086 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.233696 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-d6c34060-23da-410c-bb69-bf08dfafe80a admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105428) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:50.234971 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-d6c34060-23da-410c-bb69-bf08dfafe80a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.385953 Sep 27 04:13:50.234971 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 04:13:49 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 387 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:50.240585 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-f139c548-5953-4aed-bf6b-60e26d6cf894 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:50.255644 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-f139c548-5953-4aed-bf6b-60e26d6cf894 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018488 Sep 27 04:13:50.256165 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 22/45] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:50 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:50.266070 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f46f9cd2-aaf5-4175-9f2d-07242f84f87e admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:50.281533 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f46f9cd2-aaf5-4175-9f2d-07242f84f87e admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019529 Sep 27 04:13:50.282064 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:50 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:50.291836 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-15ac36fe-b592-44a2-bf4d-a5b14748d629 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:50.296915 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.wsgi [None req-15ac36fe-b592-44a2-bf4d-a5b14748d629 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-15ac36fe-b592-44a2-bf4d-a5b14748d629 admin admin] Request method failure captured: Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept: application/json Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept-Encoding: gzip, deflate Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Connection: keep-alive Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Content-Length: 0 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Host: 10.222.0.26 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-System-Scope: None Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: User-Agent: python-novaclient Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Auth-Token: gAAAAABjMnh9JyMhSTKtECrbxg9Pfl6ILz1dPTLpg_Yun4TvC4czgdiRIqC7JGOZyZekQ-CaMDn39feQgY9GYn-jlL-DbXke7UnnkL0Fu1h12dFecvT3M7iPStqNOZbe_otIqzIrW_7sPys3Rgap38Yfg1zFs5EgWvdVXfNP5jRDfEtTbmrB28U Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Id: None Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Name: None Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Identity-Status: Confirmed Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Is-Admin-Project: True Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Id: default Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Name: Default Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Name: admin Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Role: admin,reader,member Sep 27 04:13:50.298487 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Roles: admin,reader,member Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant: admin Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Name: admin Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User: admin Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Id: default Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Name: Default Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Name: admin Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: method: > Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: exception: Flavor 84 could not be found. Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: action_args: {'id': '84'} Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:50.300054 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:50.301242 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:50.302560 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-15ac36fe-b592-44a2-bf4d-a5b14748d629 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105429) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:50.303815 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-15ac36fe-b592-44a2-bf4d-a5b14748d629 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012177 Sep 27 04:13:50.303815 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 23/47] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 04:13:50 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:51.500190 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-f46f9cd2-aaf5-4175-9f2d-07242f84f87e admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:51.501198 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-f46f9cd2-aaf5-4175-9f2d-07242f84f87e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Sep 27 04:13:51.501589 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 25/48] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:52.019777 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-c258e26f-17fc-4fb6-81f1-e6314d3a2c1d admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:52.026043 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.wsgi [None req-c258e26f-17fc-4fb6-81f1-e6314d3a2c1d admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-c258e26f-17fc-4fb6-81f1-e6314d3a2c1d admin admin] Request method failure captured: Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept: application/json Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Accept-Encoding: gzip, deflate Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Connection: keep-alive Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Content-Length: 0 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Host: 10.222.0.26 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Openstack-System-Scope: None Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: User-Agent: python-novaclient Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Auth-Token: gAAAAABjMnh_9DFLugfSK-MmTKKcEQnCFONn1VnffDJMoAufuGWL9fzeU26KbgcI7LNICmkiSEUD2lpNXmvC70qkk1rofuQt40INAwKMXJ4sNfxNn27TxSJfXJ1EQHGqzJ59pKytnpiewbWQPi3xcO4L2zRRamMcbLgjwLXr-Ow3UxvbWL7FAQ0 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Id: None Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Domain-Name: None Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Identity-Status: Confirmed Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Is-Admin-Project: True Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Id: default Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Domain-Name: Default Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Project-Name: admin Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Role: admin,reader,member Sep 27 04:13:52.027531 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Roles: admin,reader,member Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant: admin Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-Tenant-Name: admin Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User: admin Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Id: default Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Domain-Name: Default Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: X-User-Name: admin Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: method: > Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: exception: Flavor 451 could not be found. Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: action_args: {'id': '451'} Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:52.029111 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:52.030136 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.031286 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-c258e26f-17fc-4fb6-81f1-e6314d3a2c1d admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105429) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:52.032320 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-c258e26f-17fc-4fb6-81f1-e6314d3a2c1d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.388204 Sep 27 04:13:52.032320 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 24/49] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:13:51 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 389 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:52.039257 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-0dbd51de-3cfc-4d47-a3cb-bee55c5cc44f admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:52.054037 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-0dbd51de-3cfc-4d47-a3cb-bee55c5cc44f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018857 Sep 27 04:13:52.054562 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 26/50] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:52 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:13:52.063639 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:52.078704 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018429 Sep 27 04:13:52.079214 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 25/51] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 04:13:52 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:13:52.088795 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-38c5d2b4-019e-43a1-a307-4d3873d6cb18 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:52.093632 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.wsgi [None req-38c5d2b4-019e-43a1-a307-4d3873d6cb18 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-38c5d2b4-019e-43a1-a307-4d3873d6cb18 admin admin] Request method failure captured: Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept: application/json Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Accept-Encoding: gzip, deflate Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Connection: keep-alive Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Content-Length: 0 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Host: 10.222.0.26 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-Api-Version: compute 2.92 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Openstack-System-Scope: None Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: User-Agent: python-novaclient Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Auth-Token: gAAAAABjMnh_9DFLugfSK-MmTKKcEQnCFONn1VnffDJMoAufuGWL9fzeU26KbgcI7LNICmkiSEUD2lpNXmvC70qkk1rofuQt40INAwKMXJ4sNfxNn27TxSJfXJ1EQHGqzJ59pKytnpiewbWQPi3xcO4L2zRRamMcbLgjwLXr-Ow3UxvbWL7FAQ0 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Id: None Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Domain-Name: None Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Identity-Status: Confirmed Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Is-Admin-Project: True Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Id: default Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Domain-Name: Default Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Project-Name: admin Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Role: admin,reader,member Sep 27 04:13:52.095199 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Roles: admin,reader,member Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3cc5c8e99cb64893a8d035d6aa4f3eb6", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3cc5c8e99cb64893a8d035d6aa4f3eb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant: admin Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Id: 3cc5c8e99cb64893a8d035d6aa4f3eb6 Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-Tenant-Name: admin Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User: admin Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Id: default Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Domain-Name: Default Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Id: 6190ca8b8acd4e99b228f0851e2ed5af Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: X-User-Name: admin Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: method: > Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: exception: Flavor 451 could not be found. Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: action_args: {'id': '451'} Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:13:52.097720 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:52.099491 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: ERROR nova.api.openstack.wsgi Sep 27 04:13:52.101925 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-38c5d2b4-019e-43a1-a307-4d3873d6cb18 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105428) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:13:52.104053 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-38c5d2b4-019e-43a1-a307-4d3873d6cb18 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011188 Sep 27 04:13:52.104053 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 27/52] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:13:52 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 04:13:53.306922 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:53.307971 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Sep 27 04:13:53.308370 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:53.826289 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-7267bcfa-0975-4ee1-9732-f4735b5af2d2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:53.845860 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-7267bcfa-0975-4ee1-9732-f4735b5af2d2 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.399732 Sep 27 04:13:53.846407 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 28/54] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 04:13:53 2022] POST /compute/v2.1/flavors => generated 404 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:13:55.065084 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:55.066194 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001550 Sep 27 04:13:55.066588 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:55.588752 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-129ebd34-e08a-4eba-ba02-8d3699b138bb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:55.606687 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-129ebd34-e08a-4eba-ba02-8d3699b138bb admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.400255 Sep 27 04:13:55.607232 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 29/56] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 04:13:55 2022] POST /compute/v2.1/flavors => generated 405 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:56.824822 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:56.825856 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Sep 27 04:13:56.826253 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:57.334732 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-e8fce8e8-639d-40ba-aa44-08ea1605c526 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:57.353626 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-e8fce8e8-639d-40ba-aa44-08ea1605c526 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.391665 Sep 27 04:13:57.354187 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 30/58] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 04:13:56 2022] POST /compute/v2.1/flavors => generated 407 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:13:58.585850 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:13:58.586862 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001529 Sep 27 04:13:58.587257 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 29/59] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:13:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:13:59.098513 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-9fc0803f-b92a-4583-9c29-cfd4bab8cd36 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:13:59.117380 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-9fc0803f-b92a-4583-9c29-cfd4bab8cd36 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.392340 Sep 27 04:13:59.118058 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 31/60] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 04:13:58 2022] POST /compute/v2.1/flavors => generated 409 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:14:00.352257 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:00.353259 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001410 Sep 27 04:14:00.353646 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 30/61] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:14:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:14:00.868400 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-3d8304b8-27c3-4ed1-ac01-ce58809b9f3a admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:00.888269 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-3d8304b8-27c3-4ed1-ac01-ce58809b9f3a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.399427 Sep 27 04:14:00.888805 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 32/62] 10.222.0.26 () {60 vars in 1202 bytes} [Tue Sep 27 04:14:00 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:14:57.303043 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.304637 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-8b7c5c82-941a-455a-9d77-c4ca7ab1bbbb admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Sep 27 04:14:57.305030 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:14:57.315492 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:14:57.691541 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-3c1c82cf-2930-428d-b1fc-141c8a323cc6 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.712315 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-3c1c82cf-2930-428d-b1fc-141c8a323cc6 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.401208 Sep 27 04:14:57.712855 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 33/64] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:14:57.734225 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_api:SELECT=14 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:14:57.738145 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-cba0f171-7fd4-4957-8054-0f016216eb65 admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.745529 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-cba0f171-7fd4-4957-8054-0f016216eb65 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012938 Sep 27 04:14:57.746098 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 32/65] 10.222.0.26 () {60 vars in 1283 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:14:57.757611 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-41af32ed-cac6-416c-ba73-56ee7955374a admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.764812 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-41af32ed-cac6-416c-ba73-56ee7955374a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010778 Sep 27 04:14:57.765316 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 34/66] 10.222.0.26 () {60 vars in 1286 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:14:57.777056 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.784246 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010574 Sep 27 04:14:57.784770 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 33/67] 10.222.0.26 () {60 vars in 1286 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:14:57.796157 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-cf25b0ae-298e-49a3-8a4a-527fd23e5106 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:14:57.803074 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-cf25b0ae-298e-49a3-8a4a-527fd23e5106 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010257 Sep 27 04:14:57.803587 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 35/68] 10.222.0.26 () {60 vars in 1283 bytes} [Tue Sep 27 04:14:57 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 11 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:17:59.559011 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:17:59.560093 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001511 Sep 27 04:17:59.560489 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:17:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:17:59.698574 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_api:SELECT=6 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:17:59.855225 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:17:59.859825 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:17:59.860295 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:17:59.860725 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:17:59.861426 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:17:59.861892 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:17:59.862303 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:17:59.893164 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-b580d707-3a7e-4309-8a6f-fa2e5ae1b669 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.195879 Sep 27 04:17:59.893704 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 36/70] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:17:59 2022] GET /compute/v2.1/os-services => generated 791 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:18:00.986340 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:00.987348 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Sep 27 04:18:00.987742 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:18:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:18:01.188805 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:01.192190 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:01.192660 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:01.193111 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:01.193856 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:01.194300 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:01.194710 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:01.213522 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-5071172c-5747-4040-89f2-79b858bfcfea admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.080003 Sep 27 04:18:01.214102 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 37/72] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:18:01 2022] GET /compute/v2.1/os-services => generated 791 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:18:12.696799 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:12.697819 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001410 Sep 27 04:18:12.698226 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:18:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:18:12.843737 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_cell0:SELECT=4 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:18:12.844456 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_cell1:SELECT=4 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:18:12.845057 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG dbcounter [-] [105428] Writing DB stats nova_api:SELECT=8 {{(pid=105428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:18:12.893441 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:12.896805 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:12.897270 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:12.897689 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:12.898706 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:12.899147 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:12.899552 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:12.924970 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-92309b55-afc8-4199-85f1-2dd1db2d4a22 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.086308 Sep 27 04:18:12.925480 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 38/74] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:18:12 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:18:13.992257 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG nova.api.openstack.wsgi [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] Calling method '>' {{(pid=105429) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:13.993343 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: INFO nova.api.openstack.requestlog [None req-ff06c38b-6990-4f2e-9797-b0c512af8cca admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001504 Sep 27 04:18:13.993926 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: [pid: 105429|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 04:18:13 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:18:14.182734 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG nova.api.openstack.wsgi [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Calling method '>' {{(pid=105428) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:18:14.186700 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:14.187186 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:14.187606 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:14.188302 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:18:14.188735 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:18:14.189164 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: DEBUG oslo_concurrency.lockutils [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105428) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:18:14.210049 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: INFO nova.api.openstack.requestlog [None req-9f7c1e91-f13d-4e27-9ac9-7b6ae138a97b admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.077382 Sep 27 04:18:14.210526 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: [pid: 105428|app: 0|req: 39/76] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 04:18:14 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:19:12.798504 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: Tue Sep 27 04:19:12 2022 - graceful shutdown triggered... Sep 27 04:19:12.798996 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: Gracefully killing worker 2 (pid: 105429)... Sep 27 04:19:12.799408 dv-cinder-859186-3-iscsi devstack@n-api.service[105428]: Gracefully killing worker 1 (pid: 105428)... Sep 27 04:19:12.800643 dv-cinder-859186-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 27 04:19:12.925698 dv-cinder-859186-3-iscsi devstack@n-api.service[105429]: DEBUG dbcounter [-] [105429] Writing DB stats nova_api:SELECT=4 {{(pid=105429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:19:13.799657 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: worker 1 buried after 1 seconds Sep 27 04:19:13.800404 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: worker 2 buried after 1 seconds Sep 27 04:19:13.800404 dv-cinder-859186-3-iscsi devstack@n-api.service[105427]: goodbye to uWSGI. Sep 27 04:19:13.801258 dv-cinder-859186-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 27 04:19:13.801884 dv-cinder-859186-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 27 04:19:13.804818 dv-cinder-859186-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 27 04:19:13.816249 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 27 04:19:13.818245 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 04:19:13 2022] *** Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: nodename: dv-cinder-859186-3-iscsi Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: machine: x86_64 Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: clock source: unix Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: pcre jit disabled Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: detected number of CPU cores: 4 Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: current working directory: / Sep 27 04:19:13.818547 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: detected binary path: /usr/bin/uwsgi-core Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: your processes number limit is 47680 Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: your memory page size is 4096 bytes Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: detected max file descriptor number: 2048 Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: lock engine: pthread robust mutexes Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: thunder lock: enabled Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 27 04:19:13.819253 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 04:19:13.857924 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: Python main interpreter initialized at 0x55a0447cc9c0 Sep 27 04:19:13.857924 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: python threads support enabled Sep 27 04:19:13.857924 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: your server socket listen backlog is limited to 100 connections Sep 27 04:19:13.857924 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: your mercy for graceful operations on workers is 90 seconds Sep 27 04:19:13.859002 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 04:19:13.859002 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: *** Operational MODE: preforking *** Sep 27 04:19:13.859162 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: *** uWSGI is running in multiple interpreter mode *** Sep 27 04:19:13.859162 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: spawned uWSGI master process (pid: 120404) Sep 27 04:19:13.859337 dv-cinder-859186-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 27 04:19:13.859914 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: spawned uWSGI worker 1 (pid: 120423, cores: 1) Sep 27 04:19:13.860269 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: spawned uWSGI worker 2 (pid: 120424, cores: 1) Sep 27 04:19:13.860422 dv-cinder-859186-3-iscsi devstack@n-api.service[120404]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 04:19:14.530029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 04:19:14.534652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 04:19:15.694880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:19:15.694880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: from cryptography.utils import int_from_bytes Sep 27 04:19:15.695493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:19:15.695493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: from cryptography.utils import int_from_bytes Sep 27 04:19:15.708645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:19:15.708645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: from cryptography.utils import int_from_bytes Sep 27 04:19:15.709177 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 04:19:15.709177 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: from cryptography.utils import int_from_bytes Sep 27 04:19:18.114588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 04:19:18.115131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 04:19:18.115237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 04:19:18.115237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 04:19:18.115653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 04:19:18.115653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 04:19:18.115804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 04:19:18.115891 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 04:19:18.116221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 04:19:18.116335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.116411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 04:19:18.116604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.116712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.116889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.116999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117196 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117307 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.117988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118191 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.118923 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.119886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-859186-3-iscsi {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-859186-3-iscsi {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120657 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.120956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121787 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.121937 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.122097 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.122253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.122407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.122561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.124573 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.124845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.125069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.125292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.125514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.125729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.125968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.126192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.126413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.126648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.126866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.127073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.127322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.127567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.127792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.128008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.128231 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.128451 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.128665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.128885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.129103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.129320 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-859186-3-iscsi {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.129554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-859186-3-iscsi {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.129794 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.129990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.130201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.130396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.130552 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.130738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.130895 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.131058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.131214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.131359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.132875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.133857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.134916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135228 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135621 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.135933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.136637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.136793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.136938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.137082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.137236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.137428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.137676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.137850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138011 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138481 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.138903 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.139072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.139221 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.139376 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.139563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.140493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.140657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.140805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.140953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.141580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.142595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.142771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143106 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.143875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144030 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144576 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.144881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.145997 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146609 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.146760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147756 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.147912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148068 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148519 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.148968 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149428 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.149911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.150065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.150476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.150708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.150867 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.150867 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.151156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.151320 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.151485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.151485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.156335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.157733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.157953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.158612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.160801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.160969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.161100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.161277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164582 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.164944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165330 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.165989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166692 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166839 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.166983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.167194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.167411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.167576 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.167817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.167981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.170555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.170745 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.171424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.171618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.171789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.171975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172127 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172495 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172623 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.172878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173588 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.173970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.174131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.175231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.176387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.176387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.176387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.176387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.176387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.177729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.177951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.177951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.178265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.178448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.178663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.178854 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.179837 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.180027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.180187 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.180334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.180510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.180673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.180831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.180977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.181140 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.181287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.181431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.181583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.181879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.181879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.182224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.183310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.183466 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.183619 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.183619 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.183902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.184060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.184206 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.184349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.184500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.184646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.184801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.184960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.185114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.185258 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.185401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.185545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.185688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.185871 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.186023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.186150 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.186280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.186398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.187187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.187390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.187550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.187696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 04:19:18.187895 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188055 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188616 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.188919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.189100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.191973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.192171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.192358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.192511 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.192739 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.192900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.193977 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194747 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.194900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.195962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.196114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.196285 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.196434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.196766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.196930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.197853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.198985 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.199139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.199284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.199975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.200326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.200481 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.200635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.200790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.200936 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201099 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.201874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202628 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202773 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.202918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203753 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.203904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204063 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204209 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204499 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.204859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.205980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.206094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.206458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.206625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.206772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.206970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207120 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207120 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.207716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208511 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.208825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.209023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.209023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.209321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.209476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.210081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.210257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.210580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.210741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.210888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211080 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.211851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.212917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213265 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213420 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.213905 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214756 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.214982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215129 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.215918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.216205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.216431 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.216602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.216805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.216957 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.217970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.218941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.219223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.219408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.219558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.219730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.220210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.220363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.220528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.220528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221607 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.221937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222092 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222383 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222683 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.222848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223483 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.223792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224076 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.224952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.225914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226218 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226692 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.226838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.227123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.227924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.228085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.228278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.228437 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.228690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.228862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229864 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.229864 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.230526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.230682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231202 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231504 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231697 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.231847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232327 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.232919 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.233064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.233210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.233926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.234874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.235162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.235338 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.235553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.235710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.235862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.236929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.237892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238045 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238209 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.238993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.239177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.239342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.239504 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.239655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.239863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.240023 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.240749 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.240946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.241299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.241459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.241620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.241796 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242103 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242594 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242722 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.242915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.243002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.243086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.243170 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.243255 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.243356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244019 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244317 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244498 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.244961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245141 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245311 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245494 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245680 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245810 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.245996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246626 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.246936 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247221 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247470 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.247729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.248540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.248640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.248735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.248822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.248906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.249561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.249561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.249765 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.249884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.249981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250818 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.250916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251011 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251485 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.251946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252199 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252199 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.252995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253080 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.253981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254170 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254447 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.254983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.255909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256816 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.256906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.257993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.258086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.258178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.258270 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.258363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.258974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259314 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.259937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260378 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260849 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.260938 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261905 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.261999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.262970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263335 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.263985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264080 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.264969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265724 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.265872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266030 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.266955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267030 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267483 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267554 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.267958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268197 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268793 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.268871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269537 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.269957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270028 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270099 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270258 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270740 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.270955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271097 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271753 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271753 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.271968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.272939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273694 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.273981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274445 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274606 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274606 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.274941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.275956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276319 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276817 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.276918 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277242 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277796 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.277938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278421 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.278873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279374 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.279878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280242 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.280961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.281107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.281389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.281553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.281718 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.281841 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.282884 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.283974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284075 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284608 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.284924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285244 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.285906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286530 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.286860 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287002 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287327 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287507 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287835 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.287969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288356 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.288993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289549 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289659 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.289994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290203 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.290940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291485 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291797 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.291892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292116 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292219 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292751 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.292896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293374 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.293832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294374 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294494 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.294801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.295024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.295222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.295351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.295542 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.295663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.296387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.296540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.296725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.296860 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297499 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.297873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.298821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299167 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.299804 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.300998 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301099 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301655 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.301992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302762 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.302935 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.303845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304066 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304159 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304485 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.304797 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305039 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305141 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.305779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306327 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306421 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.306950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307699 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.307928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308573 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.308990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.309947 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310820 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.310920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.311134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.311236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.311496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.312110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.312339 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.312475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.312663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.312775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.313923 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.314877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.315837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316368 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.316810 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317641 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.317982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318084 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.318934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319339 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319672 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.319899 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320625 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320843 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.320949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.321155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.321246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.321473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.321645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.321835 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322148 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322532 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.322844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323045 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323364 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.323809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324346 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324505 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.324948 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325421 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325651 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.325958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.326132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.326473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.326566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.326807 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.326908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327486 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327593 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.327902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328132 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328444 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.328851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329074 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329491 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.329818 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330165 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330480 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.330783 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331005 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331420 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.331967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.332977 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.333192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.333330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.333507 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.333608 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.333856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334173 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.334894 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335430 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.335863 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336678 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336794 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.336984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.337967 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338080 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338275 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338699 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.338888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.339896 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.340821 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341025 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341722 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.341853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342382 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342721 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.342830 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.343767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344651 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344810 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.344952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345687 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.345983 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.346220 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.346325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.346555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.346762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.346882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.347865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.348825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349672 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.349818 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.350948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351362 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351575 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.351982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352504 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.352921 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353451 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.353865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354456 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.354836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355380 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355701 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.355800 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356108 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356330 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356430 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356642 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.356956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357054 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357264 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357687 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.357939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.358042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.358251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.358858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359200 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.359879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360202 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360648 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.360991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.361084 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.361321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.361413 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.361664 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.361838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362435 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362748 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.362970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.363903 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364721 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364841 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.364941 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.365899 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366109 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366727 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.366829 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367657 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367755 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.367960 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368581 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.368989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369835 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.369930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.370866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371072 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371378 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371483 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.371814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372034 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.372777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373450 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.373850 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.374725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.374839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.374947 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.375953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376576 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.376901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377207 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.377878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378399 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.378808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.379971 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380375 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.380907 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381857 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.381955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382327 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.382897 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.383835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.384978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385085 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385381 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385731 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.385966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.386953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387268 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387433 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387576 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387751 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.387845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 04:19:18.388085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.388419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.388745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.389067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.389408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.389776 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.390095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.390417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.390737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.391057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.391374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.391710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.392043 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.392356 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.392687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.393010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.393337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.393663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.394012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.394355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.394675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.395009 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.395311 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.395632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.395949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.396273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.396591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.396908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.397231 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.397555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.397913 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.398252 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.398571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.398908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.399243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.399598 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.399918 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.400525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.400864 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.401193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.401514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.401849 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 04:19:18.402098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 04:19:18.449409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO dbcounter [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Registered counter for database nova_api Sep 27 04:19:18.463748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO dbcounter [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Registered counter for database nova_api Sep 27 04:19:18.470027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_db.sqlalchemy.engines [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.478984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writer thread running {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.487732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_db.sqlalchemy.engines [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.494363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writer thread running {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.535860 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.context [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),68475614-ed24-4318-8ce9-852b74c0d16a(cell1) {{(pid=120423) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 04:19:18.539925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:19:18.540408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:19:18.542020 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:19:18.542774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:19:18.543225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:19:18.545222 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:19:18.548450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO dbcounter [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Registered counter for database nova_cell0 Sep 27 04:19:18.551969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO dbcounter [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Registered counter for database nova_cell1 Sep 27 04:19:18.554764 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.context [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),68475614-ed24-4318-8ce9-852b74c0d16a(cell1) {{(pid=120424) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 04:19:18.558897 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:19:18.559301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_db.sqlalchemy.engines [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.559420 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:19:18.561003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:19:18.561432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_db.sqlalchemy.engines [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.561827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:19:18.562291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:19:18.564354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:19:18.566072 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writer thread running {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.566595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO dbcounter [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Registered counter for database nova_cell0 Sep 27 04:19:18.570754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO dbcounter [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Registered counter for database nova_cell1 Sep 27 04:19:18.578610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_db.sqlalchemy.engines [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.580375 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_db.sqlalchemy.engines [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.586326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writer thread running {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.588424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writer thread running {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.704364 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writer thread running {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.714185 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO dbcounter [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Registered counter for database nova_cell0 Sep 27 04:19:18.721429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_db.sqlalchemy.engines [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.725956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writer thread running {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:18.736340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO dbcounter [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Registered counter for database nova_cell0 Sep 27 04:19:18.744100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_db.sqlalchemy.engines [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 04:19:18.748732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writer thread running {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 04:19:19.046640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: WARNING keystonemiddleware.auth_token [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:19:19.053941 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: WARNING keystonemiddleware.auth_token [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:19:19.068697 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: WARNING keystonemiddleware.auth_token [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:19:19.076345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: WARNING keystonemiddleware.auth_token [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:19:19.404754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: WARNING keystonemiddleware.auth_token [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:19:19.411977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: WARNING keystonemiddleware.auth_token [None req-93f6f62e-9f25-4e1b-a253-38634b897d56 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:19:19.417453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a0447cc9c0 pid: 120423 (default app) Sep 27 04:19:19.424001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: WARNING keystonemiddleware.auth_token [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 04:19:19.430120 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: WARNING keystonemiddleware.auth_token [None req-b65d151e-90be-47ab-8fbe-2d226ce25070 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 04:19:19.436139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a0447cc9c0 pid: 120424 (default app) Sep 27 04:19:58.269383 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b44359a3-1e91-4555-93d5-e4caf45b7536 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce71aeb0d0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:19:58.337581 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:19:58.339002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:19:58.340332 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:19:58.341539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:19:59.290550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b44359a3-1e91-4555-93d5-e4caf45b7536 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/290db3b6-9404-4b64-8af4-3b249f7311b3" status: 200 len: 351 microversion: 2.1 time: 1.457208 Sep 27 04:19:59.299115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 1/1] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:19:57 2022] GET /compute/v2.1/os-volumes/290db3b6-9404-4b64-8af4-3b249f7311b3 => generated 351 bytes in 1469 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:20:01.043145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:01.044142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:01.046759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:01.050590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:01.791869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d5e04523-afe8-4694-a1e7-2cee97f57ba0 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce71aba040>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:02.461384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d5e04523-afe8-4694-a1e7-2cee97f57ba0 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d1ca2546-a52d-4a08-9982-cf53fba7e52c" status: 200 len: 351 microversion: 2.1 time: 1.426002 Sep 27 04:20:02.462660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 1/2] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:20:01 2022] GET /compute/v2.1/os-volumes/d1ca2546-a52d-4a08-9982-cf53fba7e52c => generated 351 bytes in 1429 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:04.642767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-010adce7-02fa-46b8-bc3e-e20b5615a10e tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce7190e3a0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:04.932002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-010adce7-02fa-46b8-bc3e-e20b5615a10e tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/3a6f3745-7a9d-47a8-9413-410312b01b95" status: 200 len: 351 microversion: 2.1 time: 0.293862 Sep 27 04:20:04.932795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:20:04 2022] GET /compute/v2.1/os-volumes/3a6f3745-7a9d-47a8-9413-410312b01b95 => generated 351 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:04.947405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-286492b8-751c-4539-a2cc-169a8b4e7c0a tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce71917160>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:05.209376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d8e39fb-2955-4260-a25e-d7b045ae4d10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: .version_select at 0x7fce7190e430>, body: {"keypair": {"name": "tempest-keypair-1427557109", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBACaZS+zKpdB88H4EQLpJRcFP9Edj2JQwHR+5A6uhiE4ITdHWVkcSw4JyLQ8QuCvMK5+zgfmg1so5vkiAoOmzhQusHPcHfAVHjQYsvrytwTdDzMFEq8PnJiDl4hIKSCIsQ=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:05.240879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-286492b8-751c-4539-a2cc-169a8b4e7c0a tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.298216 Sep 27 04:20:05.241589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 04:20:04 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:05.259726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-af476be5-f92e-426d-9272-0b78879c8b3c tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce71917a60>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:05.361388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-1d8e39fb-2955-4260-a25e-d7b045ae4d10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:20:05.395579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-1d8e39fb-2955-4260-a25e-d7b045ae4d10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for user f3438d850ce541cb9027ea3d5fe7dbf3 and project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:20:05.443147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-af476be5-f92e-426d-9272-0b78879c8b3c tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.190777 Sep 27 04:20:05.443959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 3/5] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 04:20:05 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:05.459148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e5e1963b-98ef-4f6b-82e8-276fbaaa72b1 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce7193e9d0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:05.536058 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1d8e39fb-2955-4260-a25e-d7b045ae4d10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.400725 Sep 27 04:20:05.547556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 3/6] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:20:05 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:05.743424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e5e1963b-98ef-4f6b-82e8-276fbaaa72b1 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.288297 Sep 27 04:20:05.745311 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 04:20:05 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:05.759177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2a1a5898-0e0c-4575-b594-37595c7f6095 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce718a50d0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:05.936248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1719297491", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "networks": [{"uuid": "90468030-e15f-4d68-bea3-d020393e070f"}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:06.019235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2a1a5898-0e0c-4575-b594-37595c7f6095 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.262228 Sep 27 04:20:06.019235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 04:20:05 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:06.034982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-df07f561-6af7-4f66-942f-20c90059aa81 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce718a5f70>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:06.131409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] validate_networks() for [('90468030-e15f-4d68-bea3-d020393e070f', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:20:06.282720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-df07f561-6af7-4f66-942f-20c90059aa81 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.254493 Sep 27 04:20:06.292712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 5/9] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 04:20:06 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:06.300305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-586b0766-6e74-4553-90b6-9c7acc82a2fe tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce7184da60>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:06.435705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-586b0766-6e74-4553-90b6-9c7acc82a2fe tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.140771 Sep 27 04:20:06.436328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 6/10] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 04:20:06 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:20:06.457704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3732536c-cdbd-48e0-8ba4-8a259d8cfcf1 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce7184dc10>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:06.585245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3732536c-cdbd-48e0-8ba4-8a259d8cfcf1 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.138170 Sep 27 04:20:06.586244 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 7/11] 10.222.0.26 () {58 vars in 1188 bytes} [Tue Sep 27 04:20:06 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:20:06.602908 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-89866436-8b58-41d6-8f68-bf319cba4aa5 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] Calling method '.version_select at 0x7fce718a5e50>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:06.830177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-89866436-8b58-41d6-8f68-bf319cba4aa5 tempest-VolumesTestJSON-1568254763 tempest-VolumesTestJSON-1568254763-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.231271 Sep 27 04:20:06.840857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 8/12] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 04:20:06 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:08.832912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:20:08.833478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:20:08.833711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:20:08.834141 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:20:08.834553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:20:08.870351 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:08.871054 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:08.874889 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:08.876422 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:08.879191 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:08.879897 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:08.999443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Getting quotas for project 7914c4b5217f411f807854ccb07e0365. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:20:09.041365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Getting quotas for user 03912e6e8a074eb1814c90158c246281 and project 7914c4b5217f411f807854ccb07e0365. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:20:09.081124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:20:09.096962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:20:09.370288 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-70cbf816-746b-4b6e-8c44-94b443eb17a0 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.583014 Sep 27 04:20:09.380180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 5/13] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:20:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 3598 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:20:09.385951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e19f5ab9-9f54-45b8-b5b8-673434f12360 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:09.398403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e19f5ab9-9f54-45b8-b5b8-673434f12360 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:09.683624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e19f5ab9-9f54-45b8-b5b8-673434f12360 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:09.718460 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e19f5ab9-9f54-45b8-b5b8-673434f12360 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1215 microversion: 2.1 time: 0.336031 Sep 27 04:20:09.737411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 6/14] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:09 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1215 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:09.808007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-63dc2095-88e4-437f-acc9-be6db695b8ac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: .version_select at 0x7fce71857e50>, body: {"keypair": {"name": "tempest-keypair-1173314002", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLdrWkpWyGZWG1saIfR0riPU2biYIascsuOX0qlmiW8RRosk+UbbxVRNzUjmuV/TlfQL2hLvJc0MJlbUuse+9m0k81TJ7mz1iiyWOaASh8A7L9++LHw3jPLFsB64qNzDkg=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:09.843467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-63dc2095-88e4-437f-acc9-be6db695b8ac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:20:09.851278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-63dc2095-88e4-437f-acc9-be6db695b8ac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for user 7c56e7333ce4450b8cbd5caeafc73947 and project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:20:09.902963 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-63dc2095-88e4-437f-acc9-be6db695b8ac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.704770 Sep 27 04:20:09.904294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 9/15] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 04:20:09 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 707 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Sep 27 04:20:10.381029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-624821173", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a72f6105-d258-45cb-a84d-18590bef93c2"}], "security_groups": [{"name": "tempest-securitygroup--331391197"}], "key_name": "tempest-keypair-1427557109", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:10.698114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] validate_networks() for [('a72f6105-d258-45cb-a84d-18590bef93c2', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:20:10.745201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8da4a5cf-63f6-4be6-8bb5-d4f180b127b3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:10.746441 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8da4a5cf-63f6-4be6-8bb5-d4f180b127b3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:11.394170 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8da4a5cf-63f6-4be6-8bb5-d4f180b127b3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:11.448652 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8da4a5cf-63f6-4be6-8bb5-d4f180b127b3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1215 microversion: 2.1 time: 0.704431 Sep 27 04:20:11.449360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 10/16] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:10 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1215 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:20:11.620462 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:20:11.620932 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:20:11.621411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:20:11.621914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:20:11.622466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:20:11.638062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:11.640823 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:11.641522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:11.642599 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:11.644610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:11.649550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:11.715128 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:20:11.724006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for user f3438d850ce541cb9027ea3d5fe7dbf3 and project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:20:11.764087 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:20:11.784830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:20:11.951926 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3270a236-ee33-42b4-8a6e-54e4f7839406 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.572902 Sep 27 04:20:11.957958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 7/17] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:20:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 1585 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:20:11.967750 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6934fa43-c1b1-483e-a449-550889bb746c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:11.968613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6934fa43-c1b1-483e-a449-550889bb746c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:12.167577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6934fa43-c1b1-483e-a449-550889bb746c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:12.182564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6934fa43-c1b1-483e-a449-550889bb746c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1237 microversion: 2.1 time: 0.219044 Sep 27 04:20:12.199904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 11/18] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:11 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1237 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:12.478991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:12.479626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:12.497045 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:12.497536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:12.498019 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:12.504331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:12.520016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:12.520692 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:12.826391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:12.839049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f1d17d11-2a7a-47e4-ac26-60fef615162f tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1219 microversion: 2.1 time: 0.365327 Sep 27 04:20:12.841781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 8/19] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:12 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1219 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:13.212280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d634775d-4ebc-406a-b92a-1a5a4fa37cdc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:13.214365 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d634775d-4ebc-406a-b92a-1a5a4fa37cdc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:13.382094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d634775d-4ebc-406a-b92a-1a5a4fa37cdc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:13.400612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d634775d-4ebc-406a-b92a-1a5a4fa37cdc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1237 microversion: 2.1 time: 0.193719 Sep 27 04:20:13.423445 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 12/20] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:13 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1237 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:13.866284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:13.867214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:13.894005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:13.906348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:13.907712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:13.910873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:13.912907 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:13.913600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:14.178682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:14.204199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c11b8ae6-3ccf-4006-82c6-9630b98717b4 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1267 microversion: 2.1 time: 0.342208 Sep 27 04:20:14.207544 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 9/21] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:13 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1267 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:14.448790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:14.449571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:14.471940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:14.474629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:14.475311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:14.477269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:14.479042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:14.480245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:14.667831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-694091439", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "50cf116a-a1a9-4ea4-9151-a8bbdd56f0e6"}], "security_groups": [{"name": "tempest-securitygroup--662136365"}], "key_name": "tempest-keypair-1173314002", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:14.930861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:14.964262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] validate_networks() for [('50cf116a-a1a9-4ea4-9151-a8bbdd56f0e6', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:20:14.979809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cd2ea734-2025-4806-b5ab-ab65f38db68b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1233 microversion: 2.1 time: 0.531057 Sep 27 04:20:14.996492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 13/22] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:14 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1233 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:15.237461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:15.238158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:15.257022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:15.260457 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:15.275893 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:15.276710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:15.277462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:15.277942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:15.536327 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:15.559997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a3ca9d76-26ee-4f8f-9141-49d36f25ec6b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1273 microversion: 2.1 time: 0.328876 Sep 27 04:20:15.563125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 14/23] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:15 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1273 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:16.012818 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:16.014781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:16.043679 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.044571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.045284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.050543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.051349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.052184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.578155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:20:16.578567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:20:16.578986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:20:16.579408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:20:16.579831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:20:16.606019 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.609094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.613181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.620512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.622597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.625425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.715635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:20:16.744202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for user 7c56e7333ce4450b8cbd5caeafc73947 and project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:20:16.769597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:20:16.771948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:20:16.802450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:16.815106 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c5666a91-3990-4921-8deb-f0dfc2b43965 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1307 microversion: 2.1 time: 0.809278 Sep 27 04:20:16.818098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 15/24] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:16 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1307 bytes in 816 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:16.831321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:16.832757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:16.848551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.849702 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.851376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.853693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:16.854194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:16.857222 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:16.913695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4ac311c2-5c36-4df8-b2ed-e425532d2eaa tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 2.250541 Sep 27 04:20:16.915418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 10/25] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 04:20:14 2022] POST /compute/v2.1/servers => generated 401 bytes in 2253 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 04:20:16.927976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:16.928935 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:17.062759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:17.075432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:17.079581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:17.082888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7f179d5c-9af0-4721-97c7-25b28c136199 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1281 microversion: 2.20 time: 0.159234 Sep 27 04:20:17.102863 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 11/26] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:16 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1281 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:17.466915 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:17.482192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f34d6d9b-60c2-4bf6-be57-d40400ad7296 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1273 microversion: 2.1 time: 0.659058 Sep 27 04:20:17.483917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 16/27] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:16 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1273 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:17.841216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:17.842353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:17.869998 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:17.880938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:17.881521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:17.888235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:17.889118 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:17.892933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:18.108103 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:18.109023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:18.350976 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:18.365914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:18.372697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:18.377482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e76870fe-c455-4fdd-8734-79179e73dfe1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1281 microversion: 2.20 time: 0.273978 Sep 27 04:20:18.380565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 17/28] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:18 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1281 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:18.420606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:18.439885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c3aa95b9-f29c-40fa-9f9d-6b60b1e83484 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1295 microversion: 2.1 time: 0.597170 Sep 27 04:20:18.446259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 12/29] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:17 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1295 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:18.510586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:18.511513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:18.536980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:18.540317 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:18.545207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:18.548352 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:18.552281 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:18.560305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:19.433668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:19.435953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:19.474873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:19.476038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:19.477840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:19.482119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:19.485365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:19.486337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:19.708655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:19.717558 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:19.719087 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:19.720221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f1ba1334-582e-4b2e-acad-017f1b748c91 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1333 microversion: 2.20 time: 0.316634 Sep 27 04:20:19.720764 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 13/30] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:19 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1333 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:19.735319 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:19.738896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:19.769528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:19.772358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:19.773260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:19.777393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:19.779252 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:19.781077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:20.163929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:20.183859 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bd41300c-a52c-4316-806e-8181a1ecfa5e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1295 microversion: 2.1 time: 0.454036 Sep 27 04:20:20.188916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 14/31] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:19 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1295 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:20.349226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:20.362841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d5a4d3f8-6dd8-41de-aa3e-2ae79baaa390 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 1.855310 Sep 27 04:20:20.363474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 18/32] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:18 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 1859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:20.754603 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:20.756121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:20.766474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:20.768015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:20.768712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:20.770321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:20.771014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:20.772012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:20.895459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:20.929572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:20.936819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:20.944797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-346ebbfa-242b-48ae-ae71-f6fe9603723c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1339 microversion: 2.20 time: 0.195446 Sep 27 04:20:20.958894 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 15/33] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:20 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1339 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:21.213468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:21.213468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:21.230690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:21.231194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:21.231639 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:21.232349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:21.232809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:21.233237 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:21.394569 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:21.395446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:21.410714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:21.411215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:21.411655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:21.413514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:21.414025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:21.414463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:22.128212 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:22.146988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-30ef018a-14a2-4260-b562-57fe10f2bf35 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.760607 Sep 27 04:20:22.154559 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 16/34] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:21 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:20:22.173574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:22.175152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:22.197937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:22.198722 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:22.199750 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:22.201330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:22.202562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:22.204069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:22.559594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:22.569412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:22.585624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:22.589191 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1c5cdf65-d159-4981-9a07-cf0e3634aab5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1339 microversion: 2.20 time: 0.432643 Sep 27 04:20:22.591337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 17/35] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:22 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1339 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:23.183679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:23.186061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:23.213341 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:23.215148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:23.217590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:23.221155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:23.222308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:23.225667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:23.655414 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:23.667745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6a3e4a19-3e58-49ca-8bac-5fcd57bbb772 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.490304 Sep 27 04:20:23.677805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 18/36] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:23 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:23.703825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:23.707892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:23.731069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:23.736713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:23.738279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:23.742785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:23.744922 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:23.745424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:23.962767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:23.971887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:23.973445 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:23.987804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c224bb5a-8216-422f-957c-ebb62d789a7d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1339 microversion: 2.20 time: 0.308306 Sep 27 04:20:23.988719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 19/37] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:23 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1339 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:24.007489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:24.022865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-820d5545-e9d6-47f1-8b9c-713a777d7fa4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 2.814407 Sep 27 04:20:24.029816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 19/38] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:21 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 2822 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:24.705715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:24.707193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:24.722393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:24.724334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:24.724978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:24.726430 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:24.727082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:24.734749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:25.018892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:25.022386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:25.032844 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:25.035876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:25.037463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:25.042543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:25.043310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:25.043952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:25.075386 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:25.089632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-41ef1ab1-974e-4f1b-bed9-03736fdcd8f5 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.388186 Sep 27 04:20:25.091186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 20/39] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:24 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:20:25.098059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:25.098938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:25.134634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:25.136810 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:25.141719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:25.143725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:25.145827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:25.147035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:25.430026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:25.447911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0ef8e575-55ee-40e5-bcea-a99846b1050e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.354502 Sep 27 04:20:25.448870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 21/40] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:25 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:25.471052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:25.481610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:25.485311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:25.489313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0ec4d45b-cd75-467f-a6ed-f7e887bf8056 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1339 microversion: 2.20 time: 0.475009 Sep 27 04:20:25.495483 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:25 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1339 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:26.118868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:26.120241 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:26.138144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.143189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.147652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.148873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.151460 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.152074 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.479059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:26.480442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:26.513060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.514473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.515778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.516667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.522019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.522605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.558331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:26.569424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-250039c9-7d02-4c36-b54b-00c09a4976f3 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.454623 Sep 27 04:20:26.581023 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 22/42] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:26 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:26.594884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:26.600230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:26.615265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.617585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.618084 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.620094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:26.621391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:26.621893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:26.667068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:27.067879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=22 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:27.082436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:27.094161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ba41fe27-6d8b-40e2-b3c5-14fe65490bab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.615527 Sep 27 04:20:27.095319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 21/43] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:26 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:27.123429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:27.144666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:27.148117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:27.152929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fca41ceb-74aa-482f-a5de-b365416b3862 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.568756 Sep 27 04:20:27.167296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:26 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:27.615054 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "355252e2-3471-4c92-95d1-adcecc4033c7", "tag": "acd21fd8-866c-4c10-9697-2fe37ef43cf1"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:27.617220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:27.618363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:27.637727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:27.638970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:27.642622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:27.643273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:27.644599 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:27.647974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:27.650464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:27.658870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:27.659381 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:27.738275 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Creating event network-changed:acd21fd8-866c-4c10-9697-2fe37ef43cf1 for instance 355252e2-3471-4c92-95d1-adcecc4033c7 on hv-cinder-85918 Sep 27 04:20:27.739172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120423) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 04:20:27.741887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9df5485a-2adf-4ce8-bebb-3d8845f8fd3b req-4bd843f5-9927-4305-bb00-16292d61d8fb service nova] Acquired lock "compute-rpcapi-router" {{(pid=120423) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 04:20:27.748116 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:27.748751 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:27.750467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:27.751331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:27.752318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:27.752993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:27.791948 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.compute.rpcapi [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 27 04:20:27.793540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] Releasing lock "compute-rpcapi-router" {{(pid=120423) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 04:20:27.851863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9a3c229b-79cf-481e-b136-fc762bfb0dbb None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.753127 Sep 27 04:20:27.853172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 22/45] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:28.040338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:28.062819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e9d04bdc-8841-4d86-ac14-6ca17fa2a616 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.449497 Sep 27 04:20:28.063748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 24/46] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:27 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:28.127597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:28.128616 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:28.142529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:28.145185 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:28.146067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:28.149911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:28.150718 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:28.151523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:28.185510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:28.187007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:28.198086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:28.205548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:28.206347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:28.207462 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:28.208192 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:28.208832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:28.250578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:28.357648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=16 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:28.442760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:28.478988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:28.504756 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:28.506219 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fc00a2a0-5186-4231-8ee7-8e194ef7e029 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.325574 Sep 27 04:20:28.512111 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 25/47] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:28 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:29.087046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:29.087642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:29.098249 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:29.098735 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:29.104920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:29.107412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:29.110652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:29.113185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:29.140379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:29.161710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ca587996-e769-4b0e-884b-1294495cc13f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 1.039373 Sep 27 04:20:29.164033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:28 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 1044 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:29.556920 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:29.557666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:29.559745 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:29.570308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:29.574581 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:29.575132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:29.576014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:29.576539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:29.578044 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:29.813661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:29.824118 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:29.828509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:29.834848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-407f51de-5253-4a4d-b397-a15f77963079 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.281613 Sep 27 04:20:29.835849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 24/49] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:29 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:29.887194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e24dba8d-ecd7-482f-9c93-79e714785fec", "tag": "ef3b9eed-fccf-490a-bc0f-2fe41f79b621"}, {"name": "network-changed", "server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:29.908857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:29.909586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:29.910218 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:29.960403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Creating event network-changed:ef3b9eed-fccf-490a-bc0f-2fe41f79b621 for instance e24dba8d-ecd7-482f-9c93-79e714785fec on hv-cinder-85918 Sep 27 04:20:29.960816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] Creating event network-changed:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:20:29.975187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-9091d871-b143-432d-adc2-6beec6b16e31 req-f980e9a0-1e6a-431a-b07d-6170a79c2878 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.095075 Sep 27 04:20:29.979394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 25/50] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:29 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:30.192969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:30.193947 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:30.204126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:30.204957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:30.205932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:30.207046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:30.207558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:30.208658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:30.230986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:30.240828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ad6d6038-b97b-419a-b28b-015956fd29aa tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 1.158702 Sep 27 04:20:30.250500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 26/51] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:29 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:30.644766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:30.653386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-41b56504-22bd-4a1f-9043-15db39d625be tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.466387 Sep 27 04:20:30.669345 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 26/52] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:30 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:30.876370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:30.877731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:30.899760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:30.904629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:30.909949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:30.909949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:30.909949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:30.909949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:31.105149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:31.114295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:31.116058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:31.117190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-61f2cc50-deb8-4e85-b9de-d6fa8681c39c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.248486 Sep 27 04:20:31.117765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 27/53] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:30 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:31.279986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:31.282583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:31.299691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:31.300456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:31.300871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:31.303400 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:31.303879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:31.306055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:31.591024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:31.602777 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-987e3c97-a8ab-404c-b0db-f061d42b4a43 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.332032 Sep 27 04:20:31.623224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 27/54] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:31 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:31.677206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:31.677953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:31.690268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:31.692718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:31.694381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:31.695356 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:31.697641 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:31.698921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:32.020237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:32.034429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a54448ba-1e32-484a-a8bf-71586e699132 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.361380 Sep 27 04:20:32.047238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 28/55] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:31 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:32.147223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:32.148233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:32.163440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:32.163944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:32.166624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:32.170957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:32.173052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:32.173463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:32.365015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:32.378223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:32.380502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:32.382458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d68dbbe8-78e9-465e-8236-03b9a2b1ae36 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.241138 Sep 27 04:20:32.383490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 28/56] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:32 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:32.631071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:32.631689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:32.633704 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:32.639729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:32.640749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:32.641185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:32.642068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:32.642739 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:32.643316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:32.824015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:32.834600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c5322426-f065-4f93-9451-912f6f8c657a tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.209410 Sep 27 04:20:32.848702 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 29/57] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:32 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:33.060992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:33.061438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:33.074696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.075422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.076048 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:33.077156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.077787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.078361 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:33.324907 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:33.333802 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1ed9d747-1277-433f-84f4-54e50a85ae42 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.277172 Sep 27 04:20:33.349475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 29/58] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:33 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:33.414928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:33.415983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:33.423920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.424639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.425198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:33.426154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.426819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.427353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:33.574442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:33.585052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:33.586967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:33.588305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9d32c919-685a-4d07-b372-0fde7b5e4357 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.179233 Sep 27 04:20:33.589207 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 30/59] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:33 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:33.858010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:33.859581 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:33.866932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.867901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.868490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:33.869337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:33.870147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:33.870773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:34.097183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:34.106041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4cc1bc53-c4b4-4079-92c7-29d4580c30e7 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.251770 Sep 27 04:20:34.111138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 30/60] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:33 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:34.359243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:34.361300 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:34.376564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:34.377481 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:34.377969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:34.378687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:34.379146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:34.379577 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:34.598597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:34.614403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:34.615013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:34.620301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b13d7ad7-5f36-4107-ae5d-2c96dfad5962 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.267631 Sep 27 04:20:34.620890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 31/61] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:34 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:34.631408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:34.638348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:34.639346 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:34.641845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:34.642704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:34.644113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:34.679226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:34.885517 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:34.896377 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:34.898248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:34.899506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-48385c24-e4b3-47fe-bb68-a6d1b0315a8a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.289007 Sep 27 04:20:34.900619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 31/62] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:34 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:35.136747 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:35.137707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:35.145976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.147183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.147831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:35.149599 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.150441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.151311 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:35.384551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:35.394979 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-668415ab-d6bd-425d-8569-efc50b9ba784 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.262723 Sep 27 04:20:35.417635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:35 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:35.645474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:35.646294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:35.652358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.653114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.653554 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:35.654269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.654913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.655759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:35.799980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:35.809812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-623fafd9-8ec5-492e-a517-eb3dcf72d227 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.168705 Sep 27 04:20:35.810586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 32/64] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:35 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:35.924230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:35.924830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:35.930704 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.931195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.931636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:35.932325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:35.932785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:35.933206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:36.134461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:36.152246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:36.154254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:36.155412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d3d50b12-b94c-4f29-b646-d553c0548573 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.234960 Sep 27 04:20:36.155959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 33/65] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:35 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:36.427443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:36.428597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:36.442401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:36.443130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:36.444014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:36.444929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:36.445629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:36.446477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:36.687722 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:36.697608 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7b1f35d4-d844-4fbc-950b-96da1aaf8398 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.274544 Sep 27 04:20:36.698566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 33/66] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:36 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:36.832417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:36.833033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:36.841340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:36.846402 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:36.849687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:36.850587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:36.852858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:36.855243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.135958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:37.145369 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-143edaf3-73f4-4872-a633-db89b57e662f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.316685 Sep 27 04:20:37.159130 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 34/67] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:36 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:37.187362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:37.188501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:37.198137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.205209 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.206071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.207082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.207708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.208299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.488977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:37.497604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:37.499761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:37.500914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-acf841ae-00d7-4cd0-9559-70ba6c41b19b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.317813 Sep 27 04:20:37.501465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 34/68] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:37 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:37.508329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "355252e2-3471-4c92-95d1-adcecc4033c7", "name": "network-vif-plugged", "status": "completed", "tag": "acd21fd8-866c-4c10-9697-2fe37ef43cf1"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:37.522313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.522797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.523262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.543864 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Creating event network-vif-plugged:acd21fd8-866c-4c10-9697-2fe37ef43cf1 for instance 355252e2-3471-4c92-95d1-adcecc4033c7 on hv-cinder-85918 Sep 27 04:20:37.544393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120424) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 04:20:37.544790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-38654003-b5dd-4b67-8872-2eb0c427ed3d req-657e5813-7cf1-4011-ae5b-749e64ed4793 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120424) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 04:20:37.545868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.546484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.547070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.547784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.548237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.548658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.557569 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.compute.rpcapi [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 27 04:20:37.558477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] Releasing lock "compute-rpcapi-router" {{(pid=120424) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 04:20:37.578945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8c18658c-0088-4acf-a6e9-b600779258d3 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.153707 Sep 27 04:20:37.579872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 35/69] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:20:37.724047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:37.726028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:37.736525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.736962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.737398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.738134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:37.738746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:37.745139 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:37.784105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:38.108040 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:38.117015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c27afe85-3d83-4655-997b-071b718965c8 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.399655 Sep 27 04:20:38.117973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 35/70] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:37 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:38.179022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:38.180378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:38.191966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:38.192883 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:38.193532 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:38.194518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:38.195217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:38.195874 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:38.410412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:38.419742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-83f03398-7577-4b66-aff1-b6a4721c50fd tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.249170 Sep 27 04:20:38.436755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 36/71] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:38 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:38.528144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:38.537004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:38.545844 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:38.546685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:38.549512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:38.550691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:38.551163 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:38.551611 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:38.724302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:38.733848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:38.735312 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:38.736943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7514a735-270a-4ce4-ab7c-08fc488d839f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.213826 Sep 27 04:20:38.747921 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 36/72] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:38 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:39.141033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:39.141646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:39.159799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.160297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.160733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.161438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.161950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.162373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.324496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:39.333789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-84d45c03-ff0a-4ced-889d-76f00003e25c tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.197082 Sep 27 04:20:39.334584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 37/73] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:39 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:39.446640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:39.447257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:39.460470 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.461353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.462120 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.462835 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.463297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.463725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.615243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:39.623841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3ce6a4da-3e50-459f-b083-bf092526f65e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.182616 Sep 27 04:20:39.624986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 37/74] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:39 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:39.778119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:39.778119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:39.789504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.790180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.790913 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.792136 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:39.792798 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:39.793511 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:39.983273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:39.993164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:39.995070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:39.996194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f68caf92-e0de-4e36-a11c-4873309c0c0f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.225155 Sep 27 04:20:39.997779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 38/75] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:39 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:40.356047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:40.356787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:40.363098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:40.363789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:40.365067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:40.366074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:40.366996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:40.367585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:40.550209 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:40.558530 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4d07158b-2795-4777-bd41-1a8cd9439750 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.206481 Sep 27 04:20:40.560083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 38/76] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:40 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:40.648349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:40.649093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:40.655262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:40.655915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:40.656555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:40.657432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:40.658015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:40.658512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:40.834383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:40.844042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6f607a39-96ae-4bc4-8877-9acf113828ad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.199299 Sep 27 04:20:40.845021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 39/77] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:40 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:41.023146 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:41.023818 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:41.030934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:41.031616 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:41.032051 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:41.032742 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:41.033189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:41.033607 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:41.173344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:41.182301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:41.183740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:41.184838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-65e78ab8-6cfd-4847-9f8c-ddf03c6f1a95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.165882 Sep 27 04:20:41.185389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 39/78] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:41 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:41.583152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:41.583763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:41.596435 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:41.597731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:41.599162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:41.600926 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:41.602931 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:41.603373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:41.794927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e24dba8d-ecd7-482f-9c93-79e714785fec", "name": "network-vif-plugged", "status": "completed", "tag": "ef3b9eed-fccf-490a-bc0f-2fe41f79b621"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:41.823984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:41.824513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:41.830098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:41.884444 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] Creating event network-vif-plugged:ef3b9eed-fccf-490a-bc0f-2fe41f79b621 for instance e24dba8d-ecd7-482f-9c93-79e714785fec on hv-cinder-85918 Sep 27 04:20:41.909165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-d15a2097-662f-4f57-a6fa-75421c299fd0 req-08033dd5-0408-4736-84bb-f1bab859e98c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120023 Sep 27 04:20:41.911714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 40/79] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:41.935746 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:41.950014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2849416d-9dd1-48ef-a6a9-97c6d2a9674b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1315 microversion: 2.1 time: 0.370010 Sep 27 04:20:41.952090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 40/80] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:41 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1315 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:41.965618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:41.966513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:42.035489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:42.039018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:42.039683 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:42.040695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:42.043531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:42.044562 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:42.211273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:42.211890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:42.231180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:42.236184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:42.236914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:42.239584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:42.241156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:42.260332 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:42.549929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:42.568756 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ecdc24fd-66c5-4222-a09c-9618e52945bc tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.641896 Sep 27 04:20:42.570945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 41/81] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:41 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:42.693308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:42.701448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:42.703235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:42.704589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-be053025-ede4-4dce-8cab-a314fc5c2aa3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.497468 Sep 27 04:20:42.718780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 41/82] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:42 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:42.988256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:42.988872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:43.008817 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.011322 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.011970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.015406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.020404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.021166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.262867 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:43.274248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-480f2947-dfb2-4dab-8343-cbe8a93c18fd tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1503 microversion: 2.1 time: 0.292719 Sep 27 04:20:43.275724 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 42/83] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:42 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1503 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:43.593693 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:43.594457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:43.609126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.609887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.610501 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.612236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.612849 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.613609 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.641674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:43.730903 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:43.731030 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:43.739523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.740018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.740457 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.741658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.744472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.745138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.831592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:43.841504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-28697672-7b51-4595-a0ae-3206fec41ef1 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1362 microversion: 2.1 time: 0.252819 Sep 27 04:20:43.842153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 42/84] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:43 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1362 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:43.957923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "name": "network-vif-plugged", "status": "completed", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:43.980169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:43.981696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:43.982486 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:43.990641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:44.000139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] Creating event network-vif-plugged:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:20:44.004593 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:44.006124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-d0eb3517-60de-4a25-ad2a-e11ff3c9612c req-9b6f5088-173b-4aca-ac27-c4e2c2c0a281 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054722 Sep 27 04:20:44.008137 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 43/85] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:44.010524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:44.012637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-23748d58-b0ba-4ab4-bfb6-3abe453f24b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.286323 Sep 27 04:20:44.013435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 43/86] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:43 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:20:44.835801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4344ec3f-fffd-47a7-b95c-4a924b0a6fb4"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:44.839650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:44.848807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:44.850007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:44.850829 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:44.851752 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:44.852360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:44.853103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:44.866475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:44.867187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:44.881434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:44.882194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:44.884207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:44.885364 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:44.888277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:44.890436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:45.216468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:45.227836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2ebb32fc-fd59-407f-a2e5-08e8dece488f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1547 microversion: 2.1 time: 0.366155 Sep 27 04:20:45.229931 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 44/87] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:44 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1547 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:45.256082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:45.257227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:45.265335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:45.266191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:45.267216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:45.268271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:45.269995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:45.270660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:45.543727 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:45.560008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:45.562587 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:45.563754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9899eb66-fd38-4a43-9aea-58b0790b863d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1406 microversion: 2.20 time: 0.332649 Sep 27 04:20:45.564324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 45/88] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:45 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1406 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:45.570675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:45.571660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:45.579021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:45.579769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:45.580539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:45.581601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:45.582292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:45.583001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:45.651081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f30533f2-e84f-4108-b9bd-b494343ea512 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "POST /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.820658 Sep 27 04:20:45.654237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 44/89] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:20:44 2022] POST /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7/os-volume_attachments => generated 194 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:45.745695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-48f06e6d-94ef-475c-9b0a-fae28798c591 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.178602 Sep 27 04:20:45.747048 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 46/90] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:20:45 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-interface => generated 284 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:46.594222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:46.594928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:46.612803 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:46.613471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:46.616545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:46.617347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:46.618110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:46.618681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:47.068442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:47.076845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:47.078842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:47.080013 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-20be116c-ec8f-4358-8727-ddb807f4a838 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1598 microversion: 2.20 time: 0.491772 Sep 27 04:20:47.080573 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 45/91] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:20:46 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1598 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:20:47.109761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:47.111339 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:47.118567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:47.119227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:47.120082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:47.121630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:47.122022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:47.122656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:47.247700 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8582a523-4331-4814-a5af-7857a9b1ce6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.143355 Sep 27 04:20:47.248472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 47/92] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 04:20:47 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-interface => generated 285 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:20:48.395902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:48.622387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e24dba8d-ecd7-482f-9c93-79e714785fec", "tag": "ef3b9eed-fccf-490a-bc0f-2fe41f79b621"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:48.650168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:48.650688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:48.655441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:48.713342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] Creating event network-changed:ef3b9eed-fccf-490a-bc0f-2fe41f79b621 for instance e24dba8d-ecd7-482f-9c93-79e714785fec on hv-cinder-85918 Sep 27 04:20:48.724564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-a36152f4-8765-4c38-8168-8892ff652db3 req-297eaa2c-f659-40f5-a3d4-4c97876bc411 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.333473 Sep 27 04:20:48.729565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 46/93] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:49.523507 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:49.545694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:49.547283 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:49.548954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:49.577650 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] Creating event network-changed:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:20:49.589447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-8bf7dd31-e08d-4824-b0c6-a59c535f30e1 req-47b05ec3-26b0-4d3c-8d3c-e9e810a10765 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071632 Sep 27 04:20:49.594979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 48/94] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:54.769154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:54.770404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:54.777736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:54.782761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:54.790475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:54.791472 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:54.791953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:54.792384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:54.820026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:20:54.965373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7a9734f8-d9c8-499f-93be-6cb9f3e45748 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "DELETE /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 204 len: 0 microversion: 2.1 time: 0.199857 Sep 27 04:20:54.967022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 47/95] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:20:54 2022] DELETE /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:20:54.980815 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:54.981688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:54.991790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:54.992344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:54.995474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:54.996431 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:54.997262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:54.997871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:55.232562 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:55.242747 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d18d7233-1825-4d7f-8568-73661c197524 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1555 microversion: 2.1 time: 0.270650 Sep 27 04:20:55.258355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 49/96] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:54 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1555 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:56.267809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:56.268527 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:56.276153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:56.278478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:56.279086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:56.279962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:56.281211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:56.281834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:56.468379 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:56.493552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0e9c046b-ca06-40cc-9d6d-94d62c7d615b tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1555 microversion: 2.1 time: 0.228780 Sep 27 04:20:56.494212 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 48/97] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:56 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1555 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:57.521130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:57.522643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:57.536685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:57.540069 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:57.541732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:57.542550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:57.548996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:57.550286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:57.560824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:20:57.568254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "355252e2-3471-4c92-95d1-adcecc4033c7", "name": "network-vif-deleted", "tag": "acd21fd8-866c-4c10-9697-2fe37ef43cf1"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:20:57.581322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:57.582133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:57.582776 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:57.613632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] Creating event network-vif-deleted:acd21fd8-866c-4c10-9697-2fe37ef43cf1 for instance 355252e2-3471-4c92-95d1-adcecc4033c7 on hv-cinder-85918 Sep 27 04:20:57.622152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-1cb42625-1cf0-4bbe-a7b9-a6eaba5c72fd req-65996e2a-d1f8-44c3-8e8d-e96f67133a3c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057885 Sep 27 04:20:57.622971 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 49/98] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:20:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:20:57.686014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03912e6e8a074eb1814c90158c246281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7914c4b5217f411f807854ccb07e0365', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:20:57.704817 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-712d1ac2-c43f-447e-805d-83fcd05295f6 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 200 len: 1513 microversion: 2.1 time: 0.190693 Sep 27 04:20:57.705414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 50/99] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:57 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 1513 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:20:58.734367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:58.735676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:58.742974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:58.749523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:58.750278 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:58.751629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:58.752318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:58.753062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:58.770908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] HTTP exception thrown: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Request method failure captured: Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 HTTP/1.1 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn1znoq7lPab4W1VfinyKYKvr0NbQqPtnTgDgsm6o-nPZiXcdykrhPCw2zFiW960HY5Tjo2wk8wyu8sF1Iho4mVb-z4Ock0jiq0LEBxIVxDDudjORBo_hXGcXKRWKaqLbBlpnZEQaNB3xw_KBHMFf-ku38TRBqHv0FUelKVNGv6fZ0 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:20:58.776100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7914c4b5217f411f807854ccb07e0365", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 03912e6e8a074eb1814c90158c246281 Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.785004 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '355252e2-3471-4c92-95d1-adcecc4033c7'} Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:20:58.785812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.787466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.788474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Returning 404 to user: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:20:58.788474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1c9cf976-1107-4a4a-ac4e-c51733f74451 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 404 len: 111 microversion: 2.1 time: 0.050212 Sep 27 04:20:58.788474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 50/100] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:20:58 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:20:58.919856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:20:58.923549 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:20:58.930063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:58.930556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:58.931109 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:58.932112 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:20:58.932737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:20:58.933365 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:20:58.952735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] HTTP exception thrown: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Request method failure captured: Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7/os-volume_attachments HTTP/1.1 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn1znoq7lPab4W1VfinyKYKvr0NbQqPtnTgDgsm6o-nPZiXcdykrhPCw2zFiW960HY5Tjo2wk8wyu8sF1Iho4mVb-z4Ock0jiq0LEBxIVxDDudjORBo_hXGcXKRWKaqLbBlpnZEQaNB3xw_KBHMFf-ku38TRBqHv0FUelKVNGv6fZ0 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:20:58.960390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7914c4b5217f411f807854ccb07e0365", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 03912e6e8a074eb1814c90158c246281 Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.961677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'server_id': '355252e2-3471-4c92-95d1-adcecc4033c7'} Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:20:58.962379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:20:58.966881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:20:58.968089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Returning 404 to user: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:20:58.968089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7531aab8-1156-4043-a184-4325f5f51b34 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.046633 Sep 27 04:20:58.968089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 51/101] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:20:58 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7/os-volume_attachments => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:21:00.227966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:00.229560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:00.239866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:00.241168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:00.241822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:00.243035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:00.243467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:00.244411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:00.276117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] HTTP exception thrown: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Request method failure captured: Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: DELETE /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 HTTP/1.1 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn1znoq7lPab4W1VfinyKYKvr0NbQqPtnTgDgsm6o-nPZiXcdykrhPCw2zFiW960HY5Tjo2wk8wyu8sF1Iho4mVb-z4Ock0jiq0LEBxIVxDDudjORBo_hXGcXKRWKaqLbBlpnZEQaNB3xw_KBHMFf-ku38TRBqHv0FUelKVNGv6fZ0 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:21:00.277602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7914c4b5217f411f807854ccb07e0365", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 03912e6e8a074eb1814c90158c246281 Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.279092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '355252e2-3471-4c92-95d1-adcecc4033c7'} Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:21:00.279760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.280940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.283063 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Returning 404 to user: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:21:00.283063 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6217465b-54fd-4d48-b8b0-c8933a9b3214 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "DELETE /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 404 len: 111 microversion: 2.1 time: 0.055345 Sep 27 04:21:00.283063 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 51/102] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:21:00 2022] DELETE /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:21:00.290306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:00.290925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] [instance: 355252e2-3471-4c92-95d1-adcecc4033c7] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:00.292892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=56 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:00.299696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:00.300246 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:00.300733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:00.301638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:00.302230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:00.302735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:00.319003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] HTTP exception thrown: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Request method failure captured: Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 HTTP/1.1 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn1znoq7lPab4W1VfinyKYKvr0NbQqPtnTgDgsm6o-nPZiXcdykrhPCw2zFiW960HY5Tjo2wk8wyu8sF1Iho4mVb-z4Ock0jiq0LEBxIVxDDudjORBo_hXGcXKRWKaqLbBlpnZEQaNB3xw_KBHMFf-ku38TRBqHv0FUelKVNGv6fZ0 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:21:00.321256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7914c4b5217f411f807854ccb07e0365", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7914c4b5217f411f807854ccb07e0365"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 7914c4b5217f411f807854ccb07e0365 Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-DeleteServersTestJSON-1833106251 Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 03912e6e8a074eb1814c90158c246281 Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-DeleteServersTestJSON-1833106251-project Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.322439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '355252e2-3471-4c92-95d1-adcecc4033c7'} Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:21:00.323070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. Sep 27 04:21:00.324125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:21:00.325273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] Returning 404 to user: Instance 355252e2-3471-4c92-95d1-adcecc4033c7 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:21:00.325273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a8d2e4bf-7057-48a7-83e2-c4abd46b2393 tempest-DeleteServersTestJSON-1833106251 tempest-DeleteServersTestJSON-1833106251-project] 10.222.0.26 "GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7" status: 404 len: 111 microversion: 2.1 time: 0.036665 Sep 27 04:21:00.325273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 52/103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:00 2022] GET /compute/v2.1/servers/355252e2-3471-4c92-95d1-adcecc4033c7 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:21:11.469881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:11.470607 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=49,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:11.632356 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-509302927", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:11.699182 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] validate_networks() for [('3b03541e-a362-4f1f-858b-b7051275090d', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:21:12.222197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:21:12.222968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:21:12.223816 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:21:12.224436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:21:12.225822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:21:12.237519 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:12.238493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:12.238932 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:12.239708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:12.240163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:12.240581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:12.258807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:12.267240 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for user 2d0ba2ae3ffe4bb9b99f1347719df411 and project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:12.281996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:21:12.284497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:21:12.353463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f5635066-5efd-4ccb-a201-dc19dc496de0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.888542 Sep 27 04:21:12.354646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 52/104] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:21:11 2022] POST /compute/v2.1/servers => generated 383 bytes in 890 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:21:12.369107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8bbc2316-bff9-4aef-994c-944c45c594b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:12.370324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8bbc2316-bff9-4aef-994c-944c45c594b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:12.372573 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=8 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:12.373834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:12.485576 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8bbc2316-bff9-4aef-994c-944c45c594b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:12.494931 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8bbc2316-bff9-4aef-994c-944c45c594b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1221 microversion: 2.1 time: 0.129662 Sep 27 04:21:12.495541 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 53/105] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:12 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1221 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:12.651440 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2117983891", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:18.873612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e5e47978-60aa-4191-b5c0-f6421a4e7e97", "tag": "b602518b-6faa-4bc5-b2ba-22e555f27363"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:18.908912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:18.913828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:18.918191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:18.932153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] Creating event network-changed:b602518b-6faa-4bc5-b2ba-22e555f27363 for instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 on hv-cinder-85918 Sep 27 04:21:18.939427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-2a6fdb85-f596-4804-aa0b-5179fc41576b req-877d13ac-7fc4-4f29-9477-c5f75b1bf52c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068750 Sep 27 04:21:18.955151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 54/106] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:22.250677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:22.251858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:22.275647 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:22.669028 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=15,nova_api:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:25.305827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] validate_networks() for [('3b03541e-a362-4f1f-858b-b7051275090d', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:21:25.898754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:21:25.899174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:21:25.899643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:21:25.900264 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:21:25.901706 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:21:25.914622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:25.916280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:25.917012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:25.917719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:25.918211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:25.918804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:25.938490 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:25.943101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for user 2d0ba2ae3ffe4bb9b99f1347719df411 and project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:25.956260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:21:25.958838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:21:26.033243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-98486ec3-32cc-4499-9690-e4af24d8d1f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 13.386412 Sep 27 04:21:26.034772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 53/107] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:21:12 2022] POST /compute/v2.1/servers => generated 383 bytes in 13389 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 04:21:26.049603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f71d62f2-e099-4a04-a624-e37a814c5898 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:26.051131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f71d62f2-e099-4a04-a624-e37a814c5898 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:26.053484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:26.282229 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f71d62f2-e099-4a04-a624-e37a814c5898 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:26.291636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f71d62f2-e099-4a04-a624-e37a814c5898 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1222 microversion: 2.1 time: 0.247590 Sep 27 04:21:26.294235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 55/108] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:26 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1222 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:27.315886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:27.317122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:27.324901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:27.325730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:27.326217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:27.327496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:27.327962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:27.328516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:27.403613 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:27.416031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fe182cc1-b4ff-40c0-bc30-c39d98203b58 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1218 microversion: 2.1 time: 0.105399 Sep 27 04:21:27.416745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 54/109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:27 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1218 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:28.439433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:28.440822 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:28.447244 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:28.447741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:28.448184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:28.448896 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:28.449357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:28.449820 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:28.588573 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:28.602821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a1dce0ef-0ddd-40dd-978d-28a870d04cda tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1274 microversion: 2.1 time: 0.166914 Sep 27 04:21:28.603627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 56/110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:28 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1274 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:29.627080 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:29.628054 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:29.642724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:29.644314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:29.645558 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:29.646320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:29.649015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:29.649546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:29.803548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:29.813639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9fa23d68-99a4-4888-bdea-fa43ff136a90 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1280 microversion: 2.1 time: 0.191255 Sep 27 04:21:29.815042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 55/111] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:29 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1280 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:30.837309 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:30.838564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:30.859062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:30.864558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:30.868943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:30.872140 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:30.873031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:30.873561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:31.248760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:31.276837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bac4f19b-c565-44c5-9c7b-00358c33e733 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.443637 Sep 27 04:21:31.277639 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 57/112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:30 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:31.399642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d8c4e5f5-b02a-42c7-937e-d41da28f5fa9", "tag": "b0f3948b-c804-4958-9c96-5fefbd31faea"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:31.439160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:31.439706 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:31.440160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:31.483107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] Creating event network-changed:b0f3948b-c804-4958-9c96-5fefbd31faea for instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 on hv-cinder-85918 Sep 27 04:21:31.499749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-b25c2a35-1254-4f5b-b9fd-de81d1f3feaa req-de193a0f-2780-4187-a0cc-c5540564c141 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.107825 Sep 27 04:21:31.500283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 56/113] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:32.304560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:32.305130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:32.314902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:32.315385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:32.317297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:32.319993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:32.320821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:32.323955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:32.658846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:32.674868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bc652853-20b2-460b-8f77-f51b1e7bf6b3 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.374052 Sep 27 04:21:32.675465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 58/114] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:32 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:33.438974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e5e47978-60aa-4191-b5c0-f6421a4e7e97", "name": "network-vif-plugged", "status": "completed", "tag": "b602518b-6faa-4bc5-b2ba-22e555f27363"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:33.468662 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:33.469149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:33.472026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:33.516272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] Creating event network-vif-plugged:b602518b-6faa-4bc5-b2ba-22e555f27363 for instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 on hv-cinder-85918 Sep 27 04:21:33.529683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-877e225f-ac4d-4450-965d-419a78c47d5d req-e8c09c8d-950e-45fc-9a4c-e668b4901c7f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096177 Sep 27 04:21:33.530805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 57/115] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:33.722769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:33.723482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:33.766648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:33.766648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:33.766648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:33.766648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:33.767927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:33.767927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:34.136328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:34.147502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ba13eddd-da48-4b6c-bff7-2c43faa4ca2f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.443620 Sep 27 04:21:34.148224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 59/116] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:33 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:35.179377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:35.180024 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:35.186427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:35.186931 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:35.187353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:35.188040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:35.188486 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:35.189093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:35.357608 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:35.367252 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-169e773c-ac7e-46c6-b90c-41ef1d1ccf15 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.192441 Sep 27 04:21:35.382035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 58/117] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:35 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:36.393096 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:36.393762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:36.400673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:36.403418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:36.404876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:36.405810 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:36.411029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:36.411695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:36.412290 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:36.735148 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:36.751679 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-43d01da5-093d-4140-9356-3f3d3d340a33 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.362709 Sep 27 04:21:36.754666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 60/118] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:36 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:37.774415 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:37.775164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:37.777168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:37.777973 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:37.784333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:37.784932 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:37.785610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:37.786533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:37.787187 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:37.787811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:37.930704 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:37.938970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-da8fea7d-267e-4c4d-bbbf-9dffa911fccf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.168383 Sep 27 04:21:37.953679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 59/119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:37 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:38.810126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8c4e5f5-b02a-42c7-937e-d41da28f5fa9", "name": "network-vif-plugged", "status": "completed", "tag": "b0f3948b-c804-4958-9c96-5fefbd31faea"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:38.820585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:38.821270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:38.821692 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:38.834135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] Creating event network-vif-plugged:b0f3948b-c804-4958-9c96-5fefbd31faea for instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 on hv-cinder-85918 Sep 27 04:21:38.839217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-1c304adf-253a-4837-8c87-e4fa4fbdf36c req-63bbc6d3-b69b-493e-a4e8-a4fcf6290944 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032980 Sep 27 04:21:38.840307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 61/120] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:38.961963 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:38.963774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:38.974026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:38.974845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:38.975838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:38.976708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:38.980963 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:38.981602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:39.157893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:39.166477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3d3c9023-7fcc-462b-b68a-9b3a9212e08f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.209771 Sep 27 04:21:39.167204 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 60/121] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:38 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:39.583724 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1463465763", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:39.681598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] validate_networks() for None {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:21:40.034613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:21:40.035296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:21:40.035515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:21:40.035918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:21:40.036329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:21:40.050127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:40.050613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:40.051638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:40.052882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:40.052882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:40.052882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:40.086094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Getting quotas for project 988a2e4b6d154a4da1ee1a1f4d696588. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:40.090550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Getting quotas for user ad63c13d1c284221a761ff3550b4c6ca and project 988a2e4b6d154a4da1ee1a1f4d696588. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:40.104226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:21:40.107384 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:21:40.192882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:40.193477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:40.195723 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4e078bea-afd6-4b71-88a9-9f4ae1f8b147 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.616061 Sep 27 04:21:40.196478 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 62/122] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:21:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 618 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 04:21:40.206282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:40.207700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:40.209001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:40.213650 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2bb52d0e-e7de-42c9-a983-0093158acf37 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:40.213650 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2bb52d0e-e7de-42c9-a983-0093158acf37 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:40.214594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:40.215237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:40.215822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:40.286227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2bb52d0e-e7de-42c9-a983-0093158acf37 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:40.297177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2bb52d0e-e7de-42c9-a983-0093158acf37 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1215 microversion: 2.1 time: 0.093093 Sep 27 04:21:40.303732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 63/123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:40 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1215 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:40.442396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:40.477027 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1afb0a06-8946-47ee-9144-878b52b6fa63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1322 microversion: 2.1 time: 0.286775 Sep 27 04:21:40.509729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 61/124] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:40 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1322 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:41.326560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:41.328251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:41.341301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.343147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.343659 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.344904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.346609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.347443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.450347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:41.460158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-24261631-d669-485b-a00f-167625cdaf8f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1211 microversion: 2.1 time: 0.140419 Sep 27 04:21:41.461481 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 64/125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:41 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1211 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:41.523787 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:41.528920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:41.558070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.558795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.559732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.560617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.562544 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.563501 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.729461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-15621ef0-714e-4c16-8296-44cef0e3c940 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'create', calling method: .version_select at 0x7fce70d0eaf0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1205956249", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL6cc4D35oD5XtakKEyC3zOfAxd71De8rvwAV95BkxGx7XdLjDaWg9G4EmtCtH8/c6Q7/1CIOtpZjipuTs7nNYYTmTcSjUZYk0UvPPGTMUJcViir0kzYMaiQ/l5KMyhRjw=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:41.747912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-15621ef0-714e-4c16-8296-44cef0e3c940 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Getting quotas for project 7e7b2ff0f0104769a550d0c0692ab713. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:41.752437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-15621ef0-714e-4c16-8296-44cef0e3c940 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Getting quotas for user 42d91d56296e4fdd917495df41401921 and project 7e7b2ff0f0104769a550d0c0692ab713. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:41.768721 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:41.783167 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-15621ef0-714e-4c16-8296-44cef0e3c940 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.058070 Sep 27 04:21:41.784756 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 65/126] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:21:41 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:41.791251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-da4fe3a2-7467-40e1-9e61-f60d89b9811c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1516 microversion: 2.1 time: 0.273630 Sep 27 04:21:41.793562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 62/127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:41 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1516 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:41.826433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:41.829151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:41.836344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.840899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.841505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.842231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:41.842728 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:41.843344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:41.869253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'flavor' on Instance uuid d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:21:41.937596 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1068176738", "imageRef": "2ed30c99-f31f-483a-b820-757a8cfe8a9c", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1205956249", "networks": [{"uuid": "8bc82a56-69c2-484f-aa08-23831b7e71a2"}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:42.013884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] validate_networks() for [('8bc82a56-69c2-484f-aa08-23831b7e71a2', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:21:42.016943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1d353f6c-8429-485f-a8a0-7a7d32770698 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9/action" status: 200 len: 32 microversion: 2.1 time: 0.196575 Sep 27 04:21:42.025915 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 66/128] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:21:41 2022] POST /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9/action => generated 32 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:21:42.033542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:42.034543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:42.055336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.055854 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.056281 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.056994 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.057450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.068000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.582564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:42.584882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:42.598604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-36265309-3c53-4243-82b0-fc4463657aa9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.569166 Sep 27 04:21:42.600620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 67/129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:42 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:42.632232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:42.636707 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:42.646934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=4 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:42.651214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.652588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.653033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.653766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.656401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.657828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.704915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:21:42.706184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:21:42.707007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f159ebf1-c453-4b5a-9746-ad7515fd1f5b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/ips" status: 200 len: 144 microversion: 2.1 time: 0.104863 Sep 27 04:21:42.707351 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:21:42.707898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:21:42.708378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:21:42.708637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 68/130] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:21:42 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/ips => generated 144 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:42.720358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:42.722687 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:42.725898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.726617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.727228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.728742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.729880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.730324 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.737276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.737870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.738306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.739010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:42.739461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:42.739933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:42.753591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Getting quotas for project 7e7b2ff0f0104769a550d0c0692ab713. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:42.757670 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Getting quotas for user 42d91d56296e4fdd917495df41401921 and project 7e7b2ff0f0104769a550d0c0692ab713. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:42.771336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:21:42.774571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2ed30c99-f31f-483a-b820-757a8cfe8a9c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:21:42.829340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f9d04007-2357-4eac-8a63-c870a5021340 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.897158 Sep 27 04:21:42.837144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 63/131] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:21:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 906 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:21:42.845918 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e1990f9d-2176-4050-960d-8bc8415ee482 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:42.846627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e1990f9d-2176-4050-960d-8bc8415ee482 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:42.919916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e1990f9d-2176-4050-960d-8bc8415ee482 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:42.929092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e1990f9d-2176-4050-960d-8bc8415ee482 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1259 microversion: 2.1 time: 0.090538 Sep 27 04:21:42.930022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 64/132] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:42 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1259 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:43.093799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:43.102988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f572859d-a537-4922-a6f4-ef183d1b64f1 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1273 microversion: 2.1 time: 0.392862 Sep 27 04:21:43.103568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 69/133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:42 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1273 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:43.625677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:43.626228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:43.637884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:43.641365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:43.644874 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:43.646201 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:43.647525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:43.649683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:43.868742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:43.876944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ff252a66-4005-4239-bb33-cd6888eb022e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.255876 Sep 27 04:21:43.889126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 65/134] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:43 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:43.955939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:43.956566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:43.965917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:43.966575 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:43.967022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:43.968415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:43.968875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:43.969303 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:44.040200 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:44.050800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-45dc374e-f01a-46cb-a531-7cf96b15c0ce tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1263 microversion: 2.1 time: 0.097704 Sep 27 04:21:44.050800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 70/135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:43 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1263 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:44.130828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:44.131465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:44.137644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:44.138193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:44.138708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:44.139439 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:44.140410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:44.140920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:44.273305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:44.281330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b9e65f42-c8b4-4250-9c89-21c4c58ecab9 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.155838 Sep 27 04:21:44.281917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 66/136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:44 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:44.900188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:44.900790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:44.906889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:44.907367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:44.907800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:44.908472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:44.908924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:44.909342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.035714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:45.043398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0412d698-f3cf-4348-aa2c-ee2c2fe691b9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.146970 Sep 27 04:21:45.044005 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 71/137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:44 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:45.073653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:45.074312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:45.081233 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.081707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.082163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.082848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.083305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.083723 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.143886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:45.152303 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-dbbdbeb0-f54f-4283-9260-9b99d8f0559c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1263 microversion: 2.1 time: 0.082390 Sep 27 04:21:45.152870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 67/138] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:45 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1263 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:45.304227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:45.304753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:45.310577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.311053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.312113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.312787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.313235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.313652 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.437722 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:45.445164 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e1f96530-c39a-42f8-8be7-bb9a4c9ee263 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.144876 Sep 27 04:21:45.445721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 72/139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:45 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:45.946557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:45.947627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:45.954371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.954855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.955284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.955965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:45.956414 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:45.957067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:45.982732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-eacef760-b2fb-4ba0-9df3-e854bd4a57d7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/ips" status: 200 len: 152 microversion: 2.20 time: 0.040220 Sep 27 04:21:45.983566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 68/140] 10.222.0.26 () {60 vars in 1289 bytes} [Tue Sep 27 04:21:45 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/ips => generated 152 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:21:46.066515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:46.067182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:46.073348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.073866 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.074524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.075405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.076010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.076591 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.180891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:46.183835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:46.197148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.198608 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.199425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.203010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.204262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.204944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.303710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:46.312975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4f7a47c9-422f-4019-8e4f-4721ea702a18 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1311 microversion: 2.1 time: 0.141140 Sep 27 04:21:46.314114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 69/141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:46 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1311 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:46.424851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:46.433880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-be4600a6-916e-4853-84a0-d91a65173a14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.371812 Sep 27 04:21:46.434373 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 73/142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:46 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:46.468096 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:46.468763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:46.476389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.477718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.478320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.479371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.480753 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.481404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.621373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:46.632129 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2849c5c6-37d7-470b-9e23-161371d3b9b7 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.168045 Sep 27 04:21:46.635453 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 70/143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:46 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:46.823241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "15636443-a500-413b-ada7-f5af04e71e1c", "tag": "5bc8f02a-52ca-41b8-b099-9368fd516a45"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:46.843408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:46.846865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:46.849121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:46.894322 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] Creating event network-changed:5bc8f02a-52ca-41b8-b099-9368fd516a45 for instance 15636443-a500-413b-ada7-f5af04e71e1c on hv-cinder-85918 Sep 27 04:21:46.901962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-ea3059f5-4ef2-4e37-8f1f-567bd039a107 req-94f44de7-93d1-409d-9a0f-135eff0c130f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084055 Sep 27 04:21:46.903074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 74/144] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:47.349821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7207a7df-705c-49f2-b0e0-6eec0c33de7d"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:47.351652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:47.352497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:47.358120 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:47.365886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.369625 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.370121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.370557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.371252 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.371990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.372591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.374101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.374718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.375286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.378732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.380175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.502703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:47.516317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-12fcdfb4-c8bd-4a12-beee-f8070caa093c tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1317 microversion: 2.1 time: 0.182296 Sep 27 04:21:47.517380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 71/145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:47 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1317 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:47.523432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:47.524147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:47.543100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.543766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.544372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.545529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.546774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.547440 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.789891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:47.802910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8ec1537a-a291-439f-857d-edada0318bce tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.283065 Sep 27 04:21:47.804148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 72/146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:47 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:47.829880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:47.833170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:47.851160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.853836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.854309 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.855345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:47.859495 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:47.860008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:47.912515 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:47.940778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-282671cb-77d0-43e9-a969-f4cf4c19bb35 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.598546 Sep 27 04:21:47.942393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 75/147] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:21:47 2022] POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments => generated 194 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:48.063015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:48.070705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-74ad76cf-b172-4b0c-8909-7aefc9d2720b tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.265217 Sep 27 04:21:48.071281 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 73/148] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:47 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:48.248483 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:48.249798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:48.257349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.257778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.258208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.258908 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.259355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.259776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.285294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lazy-loading 'flavor' on Instance uuid 333a79d8-9886-4cc6-bbd4-335bee1d98b6 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:21:48.549509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:48.550767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:48.562162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.563305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.564189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.564787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-10e1eb63-99f6-4533-903d-2a30cd616365 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/action" status: 202 len: 0 microversion: 2.20 time: 0.325926 Sep 27 04:21:48.568695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.569606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.569808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 76/149] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 04:21:48 2022] POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/action => generated 0 bytes in 335 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 04:21:48.570448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.576837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:48.577653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:48.587341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.587858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.588283 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.589165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.590137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.590882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.757671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:48.767982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:48.768392 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f3b0af20-c9c1-459c-aa5a-3051fd090434 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.227429 Sep 27 04:21:48.769079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 74/150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:48 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:48.799534 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:48.800996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:48.802210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-148bddd9-02f7-4552-bce7-689a57c58b0d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.230688 Sep 27 04:21:48.825419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 77/151] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:48 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:48.830443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:48.831777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:48.840090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.840829 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.841976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:48.843388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:48.844158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:48.844741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.094774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:49.101674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:49.109830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:49.123049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-461cfeef-2385-4fa5-8f03-9c0d0027c2f1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.297112 Sep 27 04:21:49.124505 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 75/152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:48 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:49.124961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.126769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.127425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.128160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.128624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.129086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.721429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:49.733688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cf9b9ad7-22e4-43a5-a22f-f25c2a12345f tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.641674 Sep 27 04:21:49.734605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 78/153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:49 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:49.798021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:49.798633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:49.806723 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.807560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.808103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.808913 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.809520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.810338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.835336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:49.836276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:49.850295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.851074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.851695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:49.853113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:49.855757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:49.859335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:50.067328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:50.122371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:50.132714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2c5fcdd0-4bf2-4535-b9e8-9f0730b83d47 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.339675 Sep 27 04:21:50.146616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 76/154] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:49 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:50.165530 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:50.166453 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:50.196149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:50.204798 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:50.206431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:50.211102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:50.215121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:50.222946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:50.246300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:50.256625 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:50.259009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:50.262934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3d0fc36a-c69c-458f-a47b-312f254e86e3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.435246 Sep 27 04:21:50.264885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 79/155] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:49 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:50.505795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:50.519590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-deb54b55-29f6-4e03-920d-53a66365919e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.360776 Sep 27 04:21:50.533778 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 77/156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:50 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:50.760232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:50.761009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:50.776258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:50.777527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:50.779245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:50.783187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:50.784984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:50.786788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:51.163033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:51.164781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:51.178424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:51.180410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:51.181005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:51.182713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:51.184520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:51.185120 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:51.415762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:51.425799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d6e59c5d-0c44-42ee-b862-169041e1d0ee tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.670429 Sep 27 04:21:51.431160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 80/157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:50 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:51.453517 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:51.458903 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:51.481911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:51.499564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:51.500241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:51.501191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:51.503189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:51.503716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:51.761394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=12 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:51.999544 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.008707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3aa39550-11d4-479e-af8c-b37736690cae tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.850393 Sep 27 04:21:52.009984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 78/158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:51 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:52.015537 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:52.016419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:52.035285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:52.037728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:52.039310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:52.040064 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:52.040550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:52.040971 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:52.245422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.265455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.268135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.269572 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-73bf324e-69d7-48d1-a432-e3f9f19a4527 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.835415 Sep 27 04:21:52.270979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 81/159] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:51 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:52.399844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.418592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f894f044-848f-4452-9b24-ce121edea15c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.403556 Sep 27 04:21:52.421689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 79/160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:52 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:52.457664 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:52.459201 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:52.459657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15636443-a500-413b-ada7-f5af04e71e1c", "name": "network-vif-plugged", "status": "completed", "tag": "5bc8f02a-52ca-41b8-b099-9368fd516a45"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:52.480496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:52.482463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:52.487029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:52.489300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:52.490205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:52.491172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:52.492178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:52.493156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:52.493800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:52.511118 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:52.551955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] Creating event network-vif-plugged:5bc8f02a-52ca-41b8-b099-9368fd516a45 for instance 15636443-a500-413b-ada7-f5af04e71e1c on hv-cinder-85918 Sep 27 04:21:52.569653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-dac097e7-2b63-4157-ba15-306135676f2f req-8d764355-1989-481e-9b56-d223e2acd1c9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121153 Sep 27 04:21:52.574950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 82/161] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:52.743803 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:52.925564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=8 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:52.934348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:52.945960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a3131e21-1d63-4fb4-b7ba-8b0a8741ddcf tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.490503 Sep 27 04:21:52.947591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 80/162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:52 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:53.038337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:53.046191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:53.058083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.058926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.059570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.060516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.061126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.062408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.307273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:53.308633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:53.340707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:53.345352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.353097 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.354515 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.355541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.358849 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.359681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.399801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:53.415725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ad13ead3-cc79-4265-9a58-c0508ed2b435 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.386743 Sep 27 04:21:53.417019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 83/163] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:53 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:53.442928 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:53.443578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:53.454696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.455529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.456192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.457092 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.457787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.464443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.750591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:53.782197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:53.784162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:53.789754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-886ecc0b-9589-41a4-bc35-b3a27fdf0030 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.485296 Sep 27 04:21:53.791771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 81/164] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:53 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:21:53.818671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:53.847572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:53.848567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:53.849256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:53.888937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] Creating event network-changed:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:21:53.907416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-c8149e23-6b29-40ec-8e2b-5cbd75c27d0e req-5a1a44d6-7e34-453d-a2d9-531efb0a2714 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110312 Sep 27 04:21:53.909824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 82/165] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:21:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:21:53.975828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:53.982589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:53.984621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:54.001416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a3b169bb-fbfd-446f-8519-8d2eebd8d772 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.561587 Sep 27 04:21:54.016454 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.031187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 84/166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:53 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:54.032543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.033519 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:54.035848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.037169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.037787 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:54.259422 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:54.271963 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e2db873e-9540-44e5-b034-4e990f61a669 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.301041 Sep 27 04:21:54.274037 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 83/167] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:53 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:54.444437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:54.446047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:54.456681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.459026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.462385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:54.463411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.464028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.465712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:54.787445 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:54.800405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-062a950f-2d31-42a1-b154-69b47c93184b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.359947 Sep 27 04:21:54.807051 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 85/168] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:54 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:54.816487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:54.817103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:54.831768 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.832295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.832782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:54.833522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:54.834139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:54.834796 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.044029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:55.044968 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:55.059849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.060503 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.062389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.063181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.063664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.067393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.180924 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:55.191115 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:55.193050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:55.196147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1e61756f-fff6-4b5c-836d-0a2841370678 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.382617 Sep 27 04:21:55.197842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 84/169] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:54 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:55.302489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:55.303355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:55.307767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:55.315095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.315780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.316381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.317258 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.317900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.318340 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ad0c2225-7948-4dd4-b16e-e50761f916e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.284688 Sep 27 04:21:55.319518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.319704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 86/170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:55 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:55.511484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:55.519744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3342d240-a05f-448b-a0a9-ff2fe707cea8 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.221118 Sep 27 04:21:55.520321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 85/171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:55 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:55.840584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:55.841263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:55.849849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.851166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.851779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:55.852641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:55.853499 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:55.854107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.100353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.110708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ba637801-f43d-4bda-bb54-17e5a980ff48 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.277106 Sep 27 04:21:56.129363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 87/172] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:55 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:56.228660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:56.229337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:56.243681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.245731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.246466 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.249503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.250289 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.250908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.348809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:56.349880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:56.359298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.362586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.364261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.365155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.370089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.370766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.601961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.613320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.615600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.619367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2a2044e7-c8f8-49c9-8705-ea05fb55596d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.395628 Sep 27 04:21:56.622370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 86/173] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:56 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:56.628268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:56.634727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:56.643166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.644163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.644989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.646079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:56.650113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:56.650939 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:56.708877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.730675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7b6e3893-baa2-4311-b7fb-3d8cbe2bc858 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.382642 Sep 27 04:21:56.734706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 88/174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:56 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:21:56.853820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:56.861644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-93dd849f-1943-4bb4-b792-e1f90042a146 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.237927 Sep 27 04:21:56.874736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 87/175] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:56 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:57.144145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:57.144839 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:57.151697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.152413 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.153344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.154074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.154526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.154945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.314847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:57.322456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-59d5f516-ed59-40f8-af08-7ca8d23bba68 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.188756 Sep 27 04:21:57.323021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 89/176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:57 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:57.672639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:57.673293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:57.679893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.680472 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.680983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.681782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.682339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.683046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.755383 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:57.756001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:57.762836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.763601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.764217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.765649 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.766318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.766897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.850224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:57.858644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:57.860296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:57.861636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ed744461-005a-47f3-9e4c-9adf053ac4bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.193032 Sep 27 04:21:57.862306 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 88/177] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:57 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:21:57.885622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:57.886382 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:57.892991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.893482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.893952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.894645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:57.895103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:57.895522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:57.910733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:57.933779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-86272552-56d9-411f-b5ae-e7755d9d63fc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.182066 Sep 27 04:21:57.949293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 90/178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:57 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:58.054060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:58.063220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-773a6eb1-ec5c-4bd3-b454-adde81e488f6 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.181490 Sep 27 04:21:58.064008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 89/179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:57 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:58.344863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:58.345499 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:58.351892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:58.352355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:58.352787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:58.353468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:58.353952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:58.354376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:58.482553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:58.490654 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4df75602-02a0-4c16-9cb5-312cfe4ec518 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.150163 Sep 27 04:21:58.491214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 91/180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:58 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:58.725892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1062908369", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "networks": [{"uuid": "14e40f69-7e37-43ea-ba44-fd89b3b60419"}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:21:58.779663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] validate_networks() for [('14e40f69-7e37-43ea-ba44-fd89b3b60419', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:21:58.888658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:58.889351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:58.895771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:58.896511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:58.897019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:58.897720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:58.898326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:58.898759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.159960 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.172157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.173833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.175717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b12f61d1-9396-47a6-b6e5-01061bd9646f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.291142 Sep 27 04:21:59.176394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 92/181] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:21:58 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:21:59.177463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:21:59.178083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:21:59.178735 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:21:59.179312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:21:59.179895 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:21:59.181417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:59.182168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:59.190325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.190744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.190848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.191184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.191548 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.192071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.192179 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.192531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.192959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.193079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.194259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.194802 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.210256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Getting quotas for project 14d42cf78c32468cb8bf10aa847c7cc3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:21:59.214358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Getting quotas for user 782ebdc88e21483eb3cecd00415b9b07 and project 14d42cf78c32468cb8bf10aa847c7cc3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:21:59.225533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:21:59.227875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:21:59.275617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-384fd4f7-0e82-438a-b790-f11bada774cd tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.553430 Sep 27 04:21:59.276403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 90/182] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:21:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 555 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:21:59.286579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:59.287230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:59.294147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.294640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.295135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.295842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.296376 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.296824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.319432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:21:59.333073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.340990 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-34e7f10f-8f96-45c9-ab1e-f4f75ef9e082 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.163544 Sep 27 04:21:59.341558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 93/183] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:59 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:59.346020 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-811587d3-9221-420d-a253-0f95f092e914 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:59.346907 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-811587d3-9221-420d-a253-0f95f092e914 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:59.432970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-811587d3-9221-420d-a253-0f95f092e914 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.442224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-811587d3-9221-420d-a253-0f95f092e914 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1219 microversion: 2.1 time: 0.099435 Sep 27 04:21:59.442823 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 94/184] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:59 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:59.471512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.479640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-537eea0b-2f2f-4b1a-855c-e2ed120c35ea tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.200731 Sep 27 04:21:59.480282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 91/185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:59 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:21:59.513230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:21:59.513878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:21:59.521685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.522207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.522646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.523358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:21:59.523813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:21:59.524242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:21:59.693057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:21:59.700847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b9d219eb-c104-4ec3-a540-cb76f181f474 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.191508 Sep 27 04:21:59.701409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 95/186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:21:59 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:00.202627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:00.203305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:00.209774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.210381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.210976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.211712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.212090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.212506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.365438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:00.366105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:00.379210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.380497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.381175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.382138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.382779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.383414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.391083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.399452 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.410436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.411861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-44fee7fa-935e-44de-bb44-ee0d1a7167dc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.214116 Sep 27 04:22:00.412421 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 92/187] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:00 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:00.471218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:00.472283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:00.484181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.485005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.485851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.487950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.488678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.490137 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.579085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.587374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-054f6f84-1683-46e5-a3bb-9e94a80fe1a1 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1215 microversion: 2.1 time: 0.126603 Sep 27 04:22:00.588898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 93/188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:00 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1215 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:00.593775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:00.594593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:00.601574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.602367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.603003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.603890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.604540 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.605169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.662563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.670212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2e76740f-c29c-4e32-b137-bbb0c4426bd1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.309048 Sep 27 04:22:00.670815 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 96/189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:00 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:00.727130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:00.727746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:00.733614 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.734178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.734614 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.735301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:00.735761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:00.736190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:00.741359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:00.749802 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8e75f365-1f59-4e36-8723-b632f5964bf0 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1315 microversion: 2.1 time: 0.158887 Sep 27 04:22:00.761146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 94/190] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:00 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:01.224890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.232527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-58c6470f-7161-4fd0-a3c4-db1b0739c7b6 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.509204 Sep 27 04:22:01.233073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 97/191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:00 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:01.247852 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:01.254331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.254820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.255260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.314746 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Going to try to stop instance {{(pid=120424) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 04:22:01.355139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f984836d-fb17-422b-b27e-1bfdcdf32060 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action" status: 202 len: 0 microversion: 2.1 time: 0.110986 Sep 27 04:22:01.355660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 95/192] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:22:01 2022] POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:22:01.363640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.364248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.370524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.371079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.371423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.372097 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.372551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.372966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.440598 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.441295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.447878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.448376 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.448820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.449519 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.450041 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.450476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.511452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.520367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5dad479f-9287-4c12-aac6-2167c7672893 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.160523 Sep 27 04:22:01.520926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 98/193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:01 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:01.580044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.587949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.589406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.590588 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fe2b0b60-a03b-441d-8d40-93b7a5b258bb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.154786 Sep 27 04:22:01.591163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 96/194] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:01 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:01.611588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.612129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.617940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.618420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.618852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.619524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.619971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.620387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.680808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.689191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1e194751-0974-46fb-a5f6-2b5769e9a873 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1271 microversion: 2.1 time: 0.081449 Sep 27 04:22:01.689851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 99/195] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:01 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1271 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:01.692698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.693308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.699147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.700822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.701283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.702347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.702536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.703048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.772805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.773417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.779336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.779833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.780546 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.781238 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.781695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.782155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.797909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:01.834179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.841890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3a28df44-c0b2-466b-b449-729e2ff1a23c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.152868 Sep 27 04:22:01.842601 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 97/196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:01 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:01.909604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:01.917315 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b9526ff7-1c3a-43b0-b504-aca60081adec tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1464 microversion: 2.1 time: 0.148428 Sep 27 04:22:01.917911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 100/197] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:01 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1464 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:01.963020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:01.963517 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:01.969949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.970438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.970874 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.971559 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:01.972018 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:01.972444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:01.998124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:22:02.217579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-77239f08-72b3-4f1c-ba9e-a0a4f770f936 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "DELETE /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 204 len: 0 microversion: 2.1 time: 0.257877 Sep 27 04:22:02.218329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 98/198] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:22:01 2022] DELETE /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:22:02.227000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.227733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.234016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.234500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.234952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.235647 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.236100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.236492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.255929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.256633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.263425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.264075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.265194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.265937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.266411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.266840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.367801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.376619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b8e72a66-2799-4cb4-b84e-b09d26fc2548 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1470 microversion: 2.1 time: 0.153741 Sep 27 04:22:02.377543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 101/199] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1470 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:02.397147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.404828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-397803e6-ca68-4629-8416-d459467e1d09 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.153024 Sep 27 04:22:02.405683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 99/200] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:02.544252 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.545079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.551693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.552186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.552631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.553326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.553808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.554248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.615578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.616186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.622135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.622627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.623068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.623775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.624782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.625209 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.681223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.690152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-718a0923-600d-4909-8cb7-0bb390664c51 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.150207 Sep 27 04:22:02.690716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 102/201] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:02.711778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.712424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.718249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.718746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.719186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.719880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.720341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.720764 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.753038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.761596 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.763111 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.764282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7e5ab448-f08a-4a85-9750-33ad7a34e661 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.152623 Sep 27 04:22:02.764837 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 100/202] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:02.781981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:02.789790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-edc8b2f6-2389-4b6a-a645-0ae6f529108a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1271 microversion: 2.1 time: 0.081635 Sep 27 04:22:02.790299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 103/203] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1271 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:02.865317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:02.865949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:02.871736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.872220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.872655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:02.873331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:02.873824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:02.874246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.000409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.008032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0710d3ba-3fc2-4dbd-951c-d95c87b253fb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.146393 Sep 27 04:22:03.008587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 101/204] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:02 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:03.400306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:03.400920 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:03.408085 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.408577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.409022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.409726 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.410208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.410641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.426892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:03.427508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:03.433841 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.434337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.434780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.435477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.435937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.436371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.551657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.559331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d1faf1f3-13e1-4038-9735-e53eb3a4162a tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1470 microversion: 2.1 time: 0.162949 Sep 27 04:22:03.559983 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 104/205] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:03 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1470 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:03.567253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.574945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5e862ab2-5829-43ca-ad51-1b060e7460bd tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.151893 Sep 27 04:22:03.575517 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 102/206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:03 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:03.712395 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:03.712923 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:03.718845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.719319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.719750 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.720423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.720890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.721294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.790650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:03.791257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:03.797126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.797611 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.798086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.798770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.799223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.799646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.847720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.856394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1b875b32-693d-4129-87fb-40bb39817310 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.147765 Sep 27 04:22:03.865176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 105/207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:03 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:03.865176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:03.865176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:03.879348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.880033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.880568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.881680 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:03.882142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:03.882684 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:03.939909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.949508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.951075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.952533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f30f725b-7b1f-4be8-87d8-957c854043e4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.165877 Sep 27 04:22:03.953268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 103/208] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:03 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:03.955037 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:03.963168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6fe9682e-ded4-42d2-bd10-6c5de539efba tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1271 microversion: 2.1 time: 0.104782 Sep 27 04:22:03.963664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 106/209] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:03 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1271 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:04.033653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:04.035116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:04.041277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.041827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.042274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.042968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.043595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.044032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.214163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:04.221757 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-80d34820-a6f1-4d75-a8bc-f9848083a69f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.195269 Sep 27 04:22:04.222323 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 104/210] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:04 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:04.582783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:04.583560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:04.589832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.590318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.590754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.591428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.592083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.593009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.598829 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:04.599448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:04.605678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.606193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.606628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.607310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.607759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.609008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.627189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:04.748452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:04.765293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:04.765541 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7c9aeee1-5f1b-484b-951a-477d98b0a3e3 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1470 microversion: 2.1 time: 0.178082 Sep 27 04:22:04.765541 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 107/211] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:04 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1470 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:04.765830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-17e08f25-fe1e-4fa5-987f-1da800b578b4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.164399 Sep 27 04:22:04.765830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 105/212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:04 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:04.879978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:04.880613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:04.886432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.886913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.887349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.888032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.888487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.888910 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.978361 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:04.979059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:04.985056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.985630 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.986183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:04.986965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:04.987567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:04.988057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.028029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.047407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-93ae5821-4d6e-445c-922b-6094ee993cef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.171139 Sep 27 04:22:05.050693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 108/213] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:04 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:05.058462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:05.059287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:05.067761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.068529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.069188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.070204 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.070849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.071547 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.142065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.150003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7bdf032e-6f56-4444-962f-0f679654bb5b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1271 microversion: 2.1 time: 0.097920 Sep 27 04:22:05.150770 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 109/214] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:05 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1271 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:05.151840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.161348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.162868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.164123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-88eb3bf3-2f2c-4989-b6f3-aaf75dfc462c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.189779 Sep 27 04:22:05.164681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 106/215] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:04 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:05.244359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:05.244966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:05.251334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.251816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.252255 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.252933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.253388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.253840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.382671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.391272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0afa614b-3ab5-48ba-9b00-3ed8ee40891d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.150631 Sep 27 04:22:05.391893 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 110/216] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:05 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:05.781159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:05.781791 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:05.787145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:05.787760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:05.788271 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.788758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.789193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.789917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.790371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.790794 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.793957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.794448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.794887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.795579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:05.796629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:05.797061 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:05.918326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.925596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:05.928416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-82cafa95-6307-424c-a816-347396fdfe78 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1470 microversion: 2.1 time: 0.151822 Sep 27 04:22:05.929000 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 107/217] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:05 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1470 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:05.935033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fe14edf3-26fc-401e-ab9a-29eb88f7175f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.151635 Sep 27 04:22:05.935595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 111/218] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:05 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:06.073150 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.073784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.080562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.081222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.081984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.082228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.082853 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.083465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.181955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.182790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.200462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.202877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.203408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.207773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.207773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.208763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.390649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.399182 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-68ac6fc0-939a-491b-af39-a8c6539a7588 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.329873 Sep 27 04:22:06.399984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 108/219] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:06.413959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.413959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-baa10c45-3ed4-4f45-a9f4-32362c9024b6 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1289 microversion: 2.1 time: 0.232680 Sep 27 04:22:06.422323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 112/220] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1289 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:06.434723 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.435344 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.436846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.437628 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.441970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.442473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.442916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.443620 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.444079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.444508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.445924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.446414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.447201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.447904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.448356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.448782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.729097 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.738437 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9d02854a-c861-4509-9658-b906ef37c17b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.307403 Sep 27 04:22:06.739016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 109/221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:06.880514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.893662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.895270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:06.896560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-dc253a39-b1b7-43c5-b890-37df426fea1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.473001 Sep 27 04:22:06.908846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 113/222] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:06.953567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.954225 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.959905 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:06.960345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.960501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:06.960821 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.961259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.961982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.962433 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.963022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.967128 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.968387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.969122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:06.970155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:06.970615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:06.971034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.206595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:07.215357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b7597ca0-1bd3-40ab-81b8-ec6741d1f445 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.259811 Sep 27 04:22:07.215933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 114/223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:07.227267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15636443-a500-413b-ada7-f5af04e71e1c", "name": "network-vif-deleted", "tag": "5bc8f02a-52ca-41b8-b099-9368fd516a45"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:07.242495 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.243137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.243839 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.256129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] Creating event network-vif-deleted:5bc8f02a-52ca-41b8-b099-9368fd516a45 for instance 15636443-a500-413b-ada7-f5af04e71e1c on hv-cinder-85918 Sep 27 04:22:07.263242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-031d22ce-b1f8-4ee5-9440-66ba72e1ef50 req-8e88041b-807e-443b-8614-3acb1d59587b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045799 Sep 27 04:22:07.267404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 115/224] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:07.375761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:07.383446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8eb38d15-07f6-4bd4-9838-912e69d0978e tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1470 microversion: 2.1 time: 0.434193 Sep 27 04:22:07.383956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 110/225] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:06 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1470 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:07.430121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:07.430699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:07.437075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:07.437371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.437696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:07.437897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.438312 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.438999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.439490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.439865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.443904 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.444401 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.445348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.445701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.446194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.446629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.591294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:07.591443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:07.599347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c5dd7b2d-0f5e-4232-8046-8429b94fd722 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.166766 Sep 27 04:22:07.599511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-64541f27-5a8f-40b8-9cc1-419fc15436bf tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.173796 Sep 27 04:22:07.599911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 111/226] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:07 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:07.600320 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 116/227] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:07 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:07.761781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:07.762414 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:07.768715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.769328 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.769870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.770743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.770941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.771421 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.923393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:07.924673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:07.931286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:07.933937 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.935341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.936089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:07.938209 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:07.938879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:07.939463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.007234 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.028217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-df5d281b-dce8-4394-a1a0-2655394c7160 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.270414 Sep 27 04:22:08.030993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 112/228] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:07 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:08.113596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.115959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:08.125627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.128276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.131263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c85afa36-4b4c-45a2-961f-3b7d44685583 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.212249 Sep 27 04:22:08.138166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 117/229] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:07 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:08.238591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:08.239321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:08.245279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.245927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.246441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.247230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.247760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.248261 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.408306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:08.408925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:08.415022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.415732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.416178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.416890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.417351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.417788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.451101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.451856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.452841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.453289 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lazy-loading 'fault' on Instance uuid 15636443-a500-413b-ada7-f5af04e71e1c {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:22:08.502337 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ad63c13d1c284221a761ff3550b4c6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '988a2e4b6d154a4da1ee1a1f4d696588', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.508215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.512729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d61469a7-b97d-4c37-8dc7-b80787a92206 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 200 len: 1305 microversion: 2.1 time: 0.107536 Sep 27 04:22:08.517789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 118/230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:08 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 1305 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:08.522126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2edea6d8-6e76-4475-a024-64b0cfd4945a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.287261 Sep 27 04:22:08.523408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 113/231] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:08 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:08.622840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:08.623160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:08.623490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:08.624541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:08.630165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.630658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.631106 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.631803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.632262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.632690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.635084 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.635708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.636242 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.637790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.637790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.638282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.796573 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.812180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-25ddc1ff-9de9-4564-807d-5dfc65198c3c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.192864 Sep 27 04:22:08.835191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 119/232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:08 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:08.896944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:08.906634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8ef7cdd1-30ce-4a29-be30-cd17658b0b5f tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.287380 Sep 27 04:22:08.907326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 114/233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:08 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:08.930945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ff20f8a-f72b-4c87-8c62-dbdf2714536d", "tag": "44284c30-6edf-45a3-8b70-fc4bb36bc551"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:08.940905 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:08.941379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:08.941829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:08.953532 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] Creating event network-changed:44284c30-6edf-45a3-8b70-fc4bb36bc551 for instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d on hv-cinder-85918 Sep 27 04:22:08.960542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-f9a10bce-4346-4a5e-8f9e-4ba19835d084 req-9f047872-6380-4522-a801-1412c2bed6f5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032535 Sep 27 04:22:08.961407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 120/234] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:09.053513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.054179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.064506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.065116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.068267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.069031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.069516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.071244 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.157865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.158668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.165671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.166424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.167086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.167883 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.168683 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.169122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.200155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:09.221322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:09.314680 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:09.326435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:09.330586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:09.332163 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8e47c7c1-07c2-4383-b3ac-e8b8ba4a730b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.179382 Sep 27 04:22:09.333171 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 121/235] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:09.390507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:09.398451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-311fb624-0638-43f8-a011-3a0d85e1d242 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.349005 Sep 27 04:22:09.399017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 115/236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:09.537350 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.538095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] [instance: 15636443-a500-413b-ada7-f5af04e71e1c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.540395 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:09.547255 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.547676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.547832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.550178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.550437 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.550520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.550520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.550520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.555394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.555974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.556524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.557237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.557700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.558157 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.567757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] HTTP exception thrown: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Request method failure captured: Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c HTTP/1.1 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnpO3L2pBv-To-0oMQUUsxjLkJ1QLKJ8yGk8n_60JqRp3Orx97U2KTzPDXoj9U0CdQmqA2yY1KEJr89KDLx49cs1KO6HYvqR7O1Y-IcXYhVwvyJ1zH_89sGbGFPlrtn6K5WqdrctjS8UhtQwB9p1le3jShRU-95hlq5muA-JE-ieaT8 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 988a2e4b6d154a4da1ee1a1f4d696588 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesNegativeTest-1183096272 Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:22:09.569472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/988a2e4b6d154a4da1ee1a1f4d696588"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/988a2e4b6d154a4da1ee1a1f4d696588"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_988a2e4b6d154a4da1ee1a1f4d696588", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/988a2e4b6d154a4da1ee1a1f4d696588"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesNegativeTest-1183096272 Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 988a2e4b6d154a4da1ee1a1f4d696588 Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesNegativeTest-1183096272 Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesNegativeTest-1183096272-project Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: ad63c13d1c284221a761ff3550b4c6ca Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesNegativeTest-1183096272-project Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. Sep 27 04:22:09.570992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '15636443-a500-413b-ada7-f5af04e71e1c'} Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:22:09.571881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. Sep 27 04:22:09.573132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:22:09.575584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] Returning 404 to user: Instance 15636443-a500-413b-ada7-f5af04e71e1c could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:22:09.575584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7f089ab1-164a-4989-9e81-c70b86ebce40 tempest-VolumesNegativeTest-1183096272 tempest-VolumesNegativeTest-1183096272-project] 10.222.0.26 "GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c" status: 404 len: 111 microversion: 2.1 time: 0.037476 Sep 27 04:22:09.575584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 122/237] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/15636443-a500-413b-ada7-f5af04e71e1c => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:22:09.868450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.869310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.878204 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.879022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.879709 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.880690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.881475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.882701 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.901132 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:09.935870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d7d5a1d2-dfab-404e-b061-9f8c7c9be971 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.393650 Sep 27 04:22:09.940166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 116/238] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:09.951131 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:09.951865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:09.963079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.970000 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.970815 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:09.971735 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:09.972375 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:09.972960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.068547 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.078701 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4d1419a8-82a2-4ff7-997b-e28f5f2e2ef9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.231038 Sep 27 04:22:10.096100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 123/239] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:10.124435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "name": "network-vif-plugged", "status": "completed", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:10.139746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.140434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.141042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.162516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] Creating event network-vif-plugged:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:22:10.178062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-a8876044-e9fc-4a4c-831f-c76671006dfb req-20eae453-ce0a-4f23-b813-d2fd2968572b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061318 Sep 27 04:22:10.180987 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 124/240] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:22:10.352950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.365024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:10.365911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c467e5c8-21a6-4471-9009-c67b42c3100d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.424542 Sep 27 04:22:10.366077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:10.377419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.378217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.378807 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.379118 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 117/241] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:09 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:10.379834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.380425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.380999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.424025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:10.425048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:10.441359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.442980 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.443555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.445473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.445991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.446431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.598571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.610513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.612066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.613512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-36d83f7c-00d8-4c40-9520-9b09b3bb925b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.256091 Sep 27 04:22:10.626227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 125/242] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:10 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:10.627646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:10.635558 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-de47be1c-1393-4ec3-82b1-fe7501cda738 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.217912 Sep 27 04:22:10.636086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 118/243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:10 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:10.964439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:10.965065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:10.974924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.975563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.975997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:10.976697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:10.977147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:10.977561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.105409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:11.106195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:11.113170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.113954 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.114593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.115325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.115898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.116524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.275399 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.281398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.294108 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-dc992950-6f85-4067-ad89-0045a1f6c3a6 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.192898 Sep 27 04:22:11.295154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 119/244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:11 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:11.295578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a7045c31-63ee-4a7f-9e87-8a8d02d409de tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.334585 Sep 27 04:22:11.298047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 126/245] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:10 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:11.391917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:11.392602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:11.399832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.400258 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.400681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.401508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.402000 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.402429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.535542 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.543887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-58f8a597-e2db-488c-b2f9-792dd7746707 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.156564 Sep 27 04:22:11.553568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 120/246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:11 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:11.639026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:11.639671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:11.645902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.646386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.646814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.647494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.647942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.648356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.659081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:11.660109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:11.662248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:11.670161 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.671396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.671842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.672928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:11.673281 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:11.674049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:11.814328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.825503 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.827010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:11.828389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-584676a4-5835-4d53-a1c2-06be6ae274b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.193174 Sep 27 04:22:11.829117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 127/247] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:11 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:12.020160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:12.034184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0f6192c6-cf51-49b2-8533-c8b36ffff209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.379600 Sep 27 04:22:12.034758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 121/248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:11 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:12.320688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:12.321001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:12.321663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:12.322418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:12.330339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.330969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.331593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.331757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.332244 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.332567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.332860 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.333226 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.333710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.333923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.334203 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.334786 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.351126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:12.494951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:12.508250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8225c3dd-5c62-420b-a20a-9fc79ca5882a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.191787 Sep 27 04:22:12.508815 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 122/249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:12 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:12.509843 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:12.518199 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9babfdb2-4e06-4adb-8d90-bdf5577e5a29 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1359 microversion: 2.1 time: 0.202640 Sep 27 04:22:12.518756 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 128/250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:12 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1359 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:12.566759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:12.567949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:12.574775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.578423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.578884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.579772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.580069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.580508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.765255 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:12.772958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e0718f30-1921-4cd1-a0a7-371704bec2b9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.210076 Sep 27 04:22:12.773516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 123/251] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:12 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:12.854046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:12.854651 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:12.861121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.861600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.862249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:12.862940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:12.863393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:12.863916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.007846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.015970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.017403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.018592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-58171b5c-736e-4822-a5b9-8118a88b58eb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1744 microversion: 2.20 time: 0.168443 Sep 27 04:22:13.019192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 129/252] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:12 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1744 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:13.056406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.057056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:13.063705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.064195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.064644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.065334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.065830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.066266 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.200139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.208377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-957bbcdf-354f-45a1-b5eb-fd98946024e6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.155573 Sep 27 04:22:13.210749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 124/253] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:13.531876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.532406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:13.539123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.539613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.540057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.540754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.541215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.541554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.542016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.542219 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:13.549338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.549880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.550314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.551007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.551468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.551896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.685642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.693885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-487c7652-a572-4154-ba90-a4c4b7fc9fcb tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1549 microversion: 2.1 time: 0.156141 Sep 27 04:22:13.694465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 125/254] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1549 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:13.718629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.719185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:13.727179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.727678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.728121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.728860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.729323 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.729774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.872743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.881827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4b986a7f-2f61-4aff-976a-7bbca6b3eac3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1549 microversion: 2.1 time: 0.167801 Sep 27 04:22:13.882784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 126/255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1549 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:13.895770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.896554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:13.904624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.908960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.908960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.911435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.915103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.915724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.916163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.920516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-95f76ff0-b4bf-4ae9-90f1-1e2418a23d15 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.392867 Sep 27 04:22:13.921133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 130/256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:13.926020 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:13.931469 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:13.932549 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Searching by: {'deleted': False, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713'} {{(pid=120423) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 04:22:13.939595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.940084 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.941649 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:13.942526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:13.942966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:13.943880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.092940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.098769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.multi_cell_list [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120423) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 04:22:14.107909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-83f40056-2c0b-4c06-a5c5-bf7603dd4bfa tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.219073 Sep 27 04:22:14.110349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 127/257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:14.120603 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:14.121220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:14.137701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.138238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.138686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.140093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.140574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.141172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.251436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-375a56af-09d8-4269-b4cc-f48447b63607 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.328786 Sep 27 04:22:14.252329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 131/258] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 04:22:13 2022] GET /compute/v2.1/servers => generated 343 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:14.259629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ff20f8a-f72b-4c87-8c62-dbdf2714536d", "name": "network-vif-plugged", "status": "completed", "tag": "44284c30-6edf-45a3-8b70-fc4bb36bc551"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:14.272752 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.273281 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.273717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.286525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] Creating event network-vif-plugged:44284c30-6edf-45a3-8b70-fc4bb36bc551 for instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d on hv-cinder-85918 Sep 27 04:22:14.298081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-fbea4f21-4663-47c3-a546-327ba64ddb43 req-a9082e96-19fc-41d4-be87-48f235b3ea4d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039442 Sep 27 04:22:14.299942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 132/259] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:14.305585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:14.306777 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:14.314147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.314646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.315160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.315807 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.316271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.316711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.336832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.355124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.357347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.359457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0f6d4a3d-787d-4e62-9384-53e785ef2517 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1744 microversion: 2.20 time: 0.248047 Sep 27 04:22:14.360233 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 128/260] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:14 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1744 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:14.366418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:14.367279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:14.383307 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.386561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.387076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.390951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.392181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.392686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.496740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.518578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-dcc5c574-cfc6-467a-8d79-989f93486162 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.217126 Sep 27 04:22:14.519311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 133/261] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:14 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:14.671858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:14.679412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3295111b-a41b-4e4f-851f-4c85a3f5863a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1549 microversion: 2.1 time: 0.318227 Sep 27 04:22:14.679978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 129/262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:14 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1549 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:14.942917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:14.943529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:14.949610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.951630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.951630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:14.951630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:14.951630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:14.952414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.133403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:15.134393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:15.144713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.144713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.144713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.144713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.145228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.145228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.177221 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.185621 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fd054419-974c-4a27-a0a6-570edbee936c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.246491 Sep 27 04:22:15.186524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 134/263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:14 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:15.277665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.286179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-17172bfa-8f8d-481d-a613-291aaa6298ee tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.156769 Sep 27 04:22:15.286938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 130/264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:15 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:15.384763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:15.385366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:15.391775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.392254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.392696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.393376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.393856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.394281 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.522390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.531264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.532713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.533902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a8798109-6fe1-4b80-a109-efbb2c007cc2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1744 microversion: 2.20 time: 0.153067 Sep 27 04:22:15.534463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 135/265] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:15 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:15.542264 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:15.543091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:15.549528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.550172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.550635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.551359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:15.553138 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:15.553736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:15.684668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:15.692912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-307f4d5e-3928-4acc-82bd-abbfab779dcf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.154456 Sep 27 04:22:15.693493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 131/266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:15 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:16.208787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:16.209479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:16.215978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.216540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.216974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.218280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.218740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.219160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.234041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:16.310568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "923d96f0-0aee-4f37-bd78-b81672577dbb"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:16.314460 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:16.321855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.322687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.323609 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.324581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.325151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.325594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.373494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.382288 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-491bb7d8-64f6-4ca0-890c-f4868cb89b68 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.177204 Sep 27 04:22:16.398050 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 136/267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:16 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:16.403349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:16.404222 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:16.410757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.411460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.412072 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.412951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.413627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.414081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.544801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.552517 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e5a78618-36f5-426b-8958-5def853d790b tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1319 microversion: 2.1 time: 0.153428 Sep 27 04:22:16.553080 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 137/268] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:16 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:16.557458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "name": "network-vif-plugged", "status": "completed", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:16.567049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.567528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.567978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.579963 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] Creating event network-vif-plugged:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:22:16.584667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-9baca745-1aec-4def-9221-3357a0914f19 req-cb4b40aa-ada7-4464-89e6-95141bd32da9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030520 Sep 27 04:22:16.585407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 138/269] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:16.590306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:16.590988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:16.596918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.597400 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.597859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.598614 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.599152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.599781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.739286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.747678 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.749961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.751051 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c70e3098-36ee-4ba5-ba10-37ce5215da79 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1744 microversion: 2.20 time: 0.164369 Sep 27 04:22:16.751749 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 139/270] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:16 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1744 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:16.757353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:16.758034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:16.764333 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.764854 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.765425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.766402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:16.767053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:16.767282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:16.910657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:16.920098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-aff8412d-06dd-43d4-bfa5-1d548a9b63a4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.167371 Sep 27 04:22:16.921781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 140/271] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:16 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:16.956112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5d108dc6-5ea8-43fb-b02d-bdc3eabcc552 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.649652 Sep 27 04:22:16.958597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 132/272] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:22:16 2022] POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments => generated 194 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:22:17.409290 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:17.410142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:17.417287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.417808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.418248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.418950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.419405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.419827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.577526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:17.578290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:17.589032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.589534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.590217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.590645 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:17.590929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.591400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.591831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.599989 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b485582a-ff9c-4309-b563-699fd16bb562 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.195491 Sep 27 04:22:17.600577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 141/273] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:17 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:17.765543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:17.774678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-929e5ecc-c84c-46d6-9919-d4cff6167adf tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1508 microversion: 2.1 time: 0.202245 Sep 27 04:22:17.775643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 133/274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:17 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1508 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:17.780997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:17.782117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:17.788868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.789722 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.790392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.791280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.791930 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.792563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.944783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:17.945661 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:17.946488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:17.953666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:17.954315 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.954986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.955432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.956237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:17.957304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:17.957771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:17.962336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:17.963762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e06ff511-cc0f-40f9-8976-ef9bb56a24c7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.189640 Sep 27 04:22:17.964825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 142/275] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:17 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:18.150521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:18.159838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4803479a-c991-4d59-b840-0bee1d6813a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.218531 Sep 27 04:22:18.160442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 134/276] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:17 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:18.629127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:18.629986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:18.635897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:18.636396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:18.637075 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:18.638101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:18.638730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:18.639161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:18.826037 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:18.838840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-955800b2-901b-4e5d-b10a-25cbb63bb34f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.213511 Sep 27 04:22:18.849940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 143/277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:18 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:18.992267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:18.993565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:19.009384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.009919 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.010360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:19.011052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.011666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.012254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:19.185579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:19.197068 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:19.204359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.205178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.206166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:19.206950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.207596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.208195 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:19.238813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:19.250653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:19.252413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:19.254222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-17aaea13-a8c1-46f1-a480-d80b49516f56 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.265642 Sep 27 04:22:19.255314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 135/278] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:18 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:19.387007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:19.395515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fb60171d-128c-46c8-aa5d-29f8821dd823 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.214266 Sep 27 04:22:19.396095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 144/279] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:19 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:19.863123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:19.864033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:19.871745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.872528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.873126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:19.874065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:19.874682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:19.875286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:20.022617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:20.039220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-97de4e25-b98a-4e7e-90ab-486d40e5aba9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1721 microversion: 2.1 time: 0.179828 Sep 27 04:22:20.039912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 136/280] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:19 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1721 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:20.286763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:20.287741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:20.295340 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:20.295962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:20.296462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:20.297251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:20.297707 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:20.298176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:20.419920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:20.421638 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:20.429602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:20.430978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:20.431938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:20.433395 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:20.434131 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:20.434721 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:20.457240 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:20.484733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:20.515791 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:20.518502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:20.519969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-dc0f6486-063a-4850-a8ab-47f983a13522 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.238225 Sep 27 04:22:20.531406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 145/281] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:20 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:20.647989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:20.657564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e239ec76-6010-4e28-8502-8a9f25e9a0b7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.241663 Sep 27 04:22:20.658190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 137/282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:20 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:21.066514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:21.067121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:21.073637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.074154 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.074591 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.075280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.075914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.076494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.220494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.230379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-09308c87-750a-45bd-ae0c-077265910668 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1698 microversion: 2.1 time: 0.168700 Sep 27 04:22:21.231220 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 146/283] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:21 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1698 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:21.257594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:21.264585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.265272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.266039 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.292833 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Going to try to start instance {{(pid=120424) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 04:22:21.364798 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3a39842c-8eb2-434f-99e7-8b1187b164b0 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action" status: 202 len: 0 microversion: 2.1 time: 0.110913 Sep 27 04:22:21.365736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 138/284] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:22:21 2022] POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 04:22:21.375568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:21.376514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:21.384723 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.385373 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.386057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.386826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.387372 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.387879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.549682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:21.550419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:21.558503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.559266 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.559890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.560595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.561217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.561863 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.564520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.580360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-11c32835-d7c1-497a-a069-0564d5e8f099 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1707 microversion: 2.1 time: 0.207915 Sep 27 04:22:21.584824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 147/285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:21 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1707 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:21.682743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:21.683630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:21.690002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.690714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.691313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.692198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:21.692656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:21.693267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:21.709200 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.718955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.720470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.724273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-622e5377-d439-4e4e-a7e6-0a788c6c0ccb tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.179563 Sep 27 04:22:21.724917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 139/286] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:21 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:21.868723 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:21.876722 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5826f9b3-37f4-4aa2-82f1-e9ef3cc5bd8d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.197778 Sep 27 04:22:21.890442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 148/287] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:21 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:22.605772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:22.610211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:22.618770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.619552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.620174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.621125 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.621836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.622524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.761021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:22.762633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:22.772250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:22.773387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.773997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.774805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.775866 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.776665 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.777164 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.783720 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-619b4467-8172-43fe-91ff-ed2b6275316d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1707 microversion: 2.1 time: 0.181825 Sep 27 04:22:22.785073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 140/288] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:22 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1707 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:22.902927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:22.904021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:22.911575 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.912690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.913388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.915114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:22.915815 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:22.916321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:22.972563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:22.982052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:22.984565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:22.985965 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-04e66145-dd5a-4c51-aea1-93bfcec5bf72 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.228935 Sep 27 04:22:23.002189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 149/289] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:22 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:23.198119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:23.206077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9ec3c926-04bb-48e1-b2e7-d34f116f91e7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.308650 Sep 27 04:22:23.206668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 141/290] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:22 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:23.815936 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:23.816563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:23.823644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:23.824129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:23.824564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:23.825505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:23.825931 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:23.826584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:23.984505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:24.011157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:24.021053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:24.023439 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:24.043971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bb7e26e0-c1c2-4450-8dab-34a00de61cf9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1707 microversion: 2.1 time: 0.228711 Sep 27 04:22:24.045083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 150/291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:23 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1707 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:24.045627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:24.046323 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:24.046856 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:24.047646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:24.048172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:24.048857 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:24.233889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:24.234770 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:24.241191 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:24.241367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:24.241686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:24.242356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:24.244870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:24.245525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:24.246162 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:24.255372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:24.257477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:24.258744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d9a30ed3-e886-43c8-8985-1acaa02e4ad7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.245282 Sep 27 04:22:24.274410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 142/292] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:24 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:24.434274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:24.445463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fa0d1115-4a25-4e74-850b-ed03c27099c6 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1522 microversion: 2.1 time: 0.216104 Sep 27 04:22:24.446641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 151/293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:24 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:25.072718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:25.073517 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:25.080407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.081014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.081522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.082431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.082978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.083480 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.237681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.246006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0266cf56-6b23-48e6-8fb8-06cb10bf0958 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1707 microversion: 2.1 time: 0.177388 Sep 27 04:22:25.246572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 143/294] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:25 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1707 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:25.285694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:25.286407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:25.292779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.293672 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.294335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.295264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.296527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.297176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.458972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.469411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.470644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:25.471259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:25.471565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.473607 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3b5907b6-9387-4c4e-be0d-ab3f271c8a77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.192492 Sep 27 04:22:25.474726 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 152/295] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:25 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:25.477987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.478667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.479269 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.480293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.481481 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.481964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.620734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.635253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7df44809-1868-493c-adb5-37c0def8f8b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1502 microversion: 2.1 time: 0.170609 Sep 27 04:22:25.636926 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 144/296] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:25 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1502 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:25.659439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:25.660379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:25.666901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.667771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.668392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.669261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.669715 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.670160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.814684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:25.815557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:25.817784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.819944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:25.823152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.823880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.824588 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.825340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:25.825908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:25.826563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:25.827376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-821bce20-1c2c-45f5-b547-6763c2880b17 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1514 microversion: 2.1 time: 0.172830 Sep 27 04:22:25.827947 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 153/297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:25 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1514 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:25.982066 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:25.992317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fe70c336-113e-438b-b847-34d6da7f3f67 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1595 microversion: 2.1 time: 0.181768 Sep 27 04:22:25.992915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 145/298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:25 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1595 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:26.272909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:26.273514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:26.281703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:26.282226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:26.282667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:26.283362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:26.284027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:26.284462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:26.469500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:26.488017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-74efe703-4d1e-4dc5-853d-2a66b609109d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1707 microversion: 2.1 time: 0.219042 Sep 27 04:22:26.489583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 154/299] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:26 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1707 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:26.505791 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:26.507120 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:26.518195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:26.519769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:26.520461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:26.522011 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:26.522689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:26.523756 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:26.779693 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:26.789879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:26.793527 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:26.795411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9cfd0c48-8009-464d-9400-19169bda8d80 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.297000 Sep 27 04:22:26.796502 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 146/300] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:26 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:27.279032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:27.282358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:27.290983 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.291820 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.294192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.295419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.314858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.317227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.361562 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'flavor' on Instance uuid e5e47978-60aa-4191-b5c0-f6421a4e7e97 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:22:27.527072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:27.528419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:27.539902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.545175 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.548903 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.550151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.550651 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8cd9d3a3-16d7-44ca-958d-8cf8e76711ee tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/action" status: 200 len: 32 microversion: 2.1 time: 0.273349 Sep 27 04:22:27.554166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 155/301] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:22:27 2022] POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/action => generated 32 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:22:27.555012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.555616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.567031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:27.567927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:27.587777 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.588526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.589145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.590107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.590847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.591428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.804464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:27.818207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d4821ede-7326-46bf-a166-0495c56875db tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.256561 Sep 27 04:22:27.819049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 156/302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:27 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:27.824672 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:27.825779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:27.833150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.833925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.834532 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:27.835387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:27.836000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:27.836566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:28.025909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:28.042223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:28.043674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:28.050427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fa4d8a03-f9de-4bbe-b7b8-ed94c3ec787b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.229263 Sep 27 04:22:28.051394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 157/303] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:27 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:28.182157 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:28.208847 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1d52150d-86d0-4e77-9fc9-aaabb8717eee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1711 microversion: 2.1 time: 0.699620 Sep 27 04:22:28.211956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 147/304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:27 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1711 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:28.844234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:28.845575 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:28.855551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:28.856060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:28.856508 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:28.857225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:28.857685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:28.858657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:28.995624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:29.003735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ccc3e9fb-70cf-4b48-ad15-4870924d4e6b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.163573 Sep 27 04:22:29.004303 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 158/305] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:28 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:29.076772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:29.077930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:29.084496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:29.085160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:29.085786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:29.086721 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:29.087064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:29.087291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:29.087945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:29.098671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:29.099065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:29.099497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:29.111880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] Creating event network-changed:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:22:29.118391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-d53b2b73-75ee-4f7e-bde6-81941c5668e3 req-6ff0dbb0-62e4-4a55-84d1-27a9cfe9319f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034671 Sep 27 04:22:29.122186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 159/306] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:22:29.359607 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:29.379234 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:29.381696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:29.383410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9fa1b78c-4324-498c-87a0-64f9996b10b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1739 microversion: 2.20 time: 0.310416 Sep 27 04:22:29.431112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 148/307] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:29 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1739 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:30.031623 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:30.033844 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:30.043288 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.043961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.044476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.045768 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.046436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.047217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.314969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:30.334765 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3804dd8c-77d1-42a4-b359-8a3e442e2393 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.306938 Sep 27 04:22:30.335529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 160/308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:30 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:30.456539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:30.457805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:30.464444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.464991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.465658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.466689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.467328 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.467797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.561426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1666472914"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:30.562141 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:30.569421 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.571086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.571625 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.572408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:30.572876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:30.573304 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:30.621605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:30.630630 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:30.633029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:30.634059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-38f26bf8-43c8-4350-9015-646637f046d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1721 microversion: 2.20 time: 0.181495 Sep 27 04:22:30.646059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 149/309] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:30 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1721 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:30.720594 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.network.security_group_api [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Adding security group e17d595b-0c19-4926-adc1-40062a9006bc to port 9d990fa5-a24c-4b14-8c63-0e0506b80887 Sep 27 04:22:31.050095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b2111d28-e3a3-49f6-a25b-22417922752b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.492556 Sep 27 04:22:31.054197 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 161/310] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:22:30 2022] POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action => generated 0 bytes in 498 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 04:22:31.060508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:31.061287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:31.073361 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.073896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.075020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.075903 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.080285 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:31.080964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.081590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.102272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.114362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.114949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.134174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] Creating event network-changed:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:22:31.140732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-5de45d18-7cee-45bb-915c-9688bbc71b2d req-b973cd79-a36a-4133-b438-8fc982fe4878 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070077 Sep 27 04:22:31.141819 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 162/311] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:31.246417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:31.255418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-787ebb4f-b200-43fc-9e53-6a366c888e02 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1761 microversion: 2.1 time: 0.198685 Sep 27 04:22:31.256066 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 150/312] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:31 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1761 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:31.361406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:31.362045 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:31.368318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.368827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.369287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.370059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.370497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.370974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.511467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:31.519859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6f13f331-6441-421c-9dd4-b3b864a7116d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.163133 Sep 27 04:22:31.520427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 163/313] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:31 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:31.661067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:31.661680 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:31.668749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.669241 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.669683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.670401 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:31.671181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:31.671617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:31.810884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:31.819603 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:31.821056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:31.822254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ea7ee339-8f46-488a-bdd6-b8d053c27c6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1721 microversion: 2.20 time: 0.165160 Sep 27 04:22:31.822804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 151/314] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:31 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:32.546863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:32.548567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:32.558279 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:32.558858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:32.559412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:32.560208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:32.560782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:32.561998 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:32.764805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:32.772847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ee627dc1-f01c-4c10-8c87-dcdca4505c98 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.231718 Sep 27 04:22:32.773418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 164/315] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:32 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:32.848571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:32.849359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:32.855782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:32.856301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:32.856740 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:32.857429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:32.857915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:32.858973 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:32.991181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.005505 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.007003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.008205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-aba08c29-33e5-49ef-b795-8fcffcb97c95 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1724 microversion: 2.20 time: 0.163569 Sep 27 04:22:33.008761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 152/316] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:32 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:33.034953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b6d46837-3522-41ab-8103-74de1fa63231"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:33.037799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:33.044904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.045405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.045920 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.046833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.047456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.048207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.407754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3c0f0eb0-d62b-44ee-8e2e-cf183c97a9fe tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.377895 Sep 27 04:22:33.408644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 165/317] 10.222.0.26 () {62 vars in 1364 bytes} [Tue Sep 27 04:22:33 2022] POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments => generated 188 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:22:33.549634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:33.550267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:33.557336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.557472 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.558114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.558949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.559560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.560377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.564605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:33.611229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120424) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 04:22:33.681708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-aa144271-f23b-4e26-84bd-bc161866cb41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/action" status: 202 len: 0 microversion: 2.20 time: 0.137000 Sep 27 04:22:33.682656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 153/318] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 04:22:33 2022] POST /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/action => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 04:22:33.692859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:33.693491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:33.699872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:33.702901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.703299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.703938 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.704638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.705693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.706575 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.798159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:33.798764 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:33.804748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.805233 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.808566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.809286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:33.809770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:33.810202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:33.882987 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.893409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.898002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.902047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ae49c883-7edb-406c-8a45-97da86053e7a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1750 microversion: 2.20 time: 0.214275 Sep 27 04:22:33.905784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 166/319] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:33 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1750 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:33.969705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:33.980127 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f779abd1-e82d-4f83-8b2b-8eada0eef69e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.186305 Sep 27 04:22:33.981102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 154/320] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:33 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:34.068894 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "name": "network-vif-unplugged", "status": "completed", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:34.082046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:34.084078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:34.085117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:34.098259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.compute.server_external_events [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] Unable to find a host for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6. Dropping event network-vif-unplugged {{(pid=120423) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 04:22:34.099440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-d92463bf-c873-4082-bc12-d25997b0ad1c req-f75dba50-5903-43f4-aa32-b5b3f48a6531 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.034595 Sep 27 04:22:34.100247 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 167/321] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:22:34 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 36 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 27 04:22:34.942738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:34.943414 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:34.952359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:34.953042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:34.953684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:34.954708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:34.955471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:34.956184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:35.004605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:35.005717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:35.012795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:35.013311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:35.013985 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:35.014882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:35.015527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:35.016214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:35.036759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=6,nova_cell1:INSERT=4 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:35.113293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:35.122639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:35.124202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:35.125676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-27e05278-cb63-4bb9-b4df-aa483017ec3e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1750 microversion: 2.20 time: 0.187225 Sep 27 04:22:35.126492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 155/322] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:34 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1750 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:35.180398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:35.188890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6178d878-578d-4539-b044-1dc43b66cb7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.188655 Sep 27 04:22:35.189467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 168/323] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:34 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:36.151044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:36.152542 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:36.159718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:36.160449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:36.161173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:36.162217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:36.162708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:36.163303 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:36.212075 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:36.212700 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:36.224593 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:36.225076 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:36.225509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:36.226334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:36.232350 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:36.232913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:36.329588 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:36.339737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:36.342095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:36.343990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0574f791-3343-475f-b6c4-301edc41fc23 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.197446 Sep 27 04:22:36.344579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 156/324] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:36 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:36.384342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:36.395393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5d02bb48-f254-42a9-a989-3adca6fdb36f tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.187644 Sep 27 04:22:36.395962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 169/325] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:36 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:37.370632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:37.371257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:37.378134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:37.378626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:37.379065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:37.379752 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:37.380214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:37.380646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:37.419040 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:37.419692 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:37.426754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:37.427246 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:37.427686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:37.428387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:37.428847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:37.429275 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:37.511341 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:37.520102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:37.521761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:37.522994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a44c2954-c20b-47fe-8665-31dff83b0f14 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.156295 Sep 27 04:22:37.523627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 157/326] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:37 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:37.555390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:37.563105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e44dee47-3396-47fc-92b6-9a4abe518b46 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.147814 Sep 27 04:22:37.563662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 170/327] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:37 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:38.548868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:38.549476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:38.556081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:38.556566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:38.557003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:38.557706 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:38.558221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:38.558642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:38.584988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:38.585605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:38.592127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:38.592659 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:38.593644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:38.594354 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:38.594805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:38.595224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:38.685964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:38.694637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:38.696061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:38.697224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5f688f71-a167-4c46-9137-16c0bb796707 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.152274 Sep 27 04:22:38.697999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 158/328] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:38 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:38.765947 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:38.773501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-dbbf8af7-fc05-4c06-87af-458114ee2397 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.192391 Sep 27 04:22:38.774111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 171/329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:38 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:39.723438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:39.724052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:39.730548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:39.731031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:39.731481 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:39.732160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:39.732621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:39.733045 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:39.798165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:39.799489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:39.806064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:39.806562 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:39.807001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:39.807701 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:39.808156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:39.808578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:39.871660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:39.883037 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:39.885579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:39.886847 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9bff6cc7-48a2-4be3-beac-1a44c58750b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.167304 Sep 27 04:22:39.887417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 159/330] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:39 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:40.009536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:40.017921 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-da33f60d-274c-485f-bc14-89506a90478d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.224193 Sep 27 04:22:40.018496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 172/331] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:39 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:40.915441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:40.915967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:40.922788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:40.923264 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:40.923696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:40.924370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:40.924819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:40.925237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:41.042240 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:41.042878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:41.051003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:41.051494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:41.052131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:41.053127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:41.053762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:41.054413 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:41.058576 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:41.074092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:41.076285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:41.080257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a374edcb-f6a5-4456-9e50-2de47d31ecde tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.167554 Sep 27 04:22:41.081605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 160/332] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:40 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:41.436872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:41.444771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5c745f8b-a142-45fb-99b1-31df628bef9a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.406760 Sep 27 04:22:41.445448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 173/333] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:41 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:42.107865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:42.108480 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:42.111081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:42.116447 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:42.117122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:42.117566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:42.118318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:42.118866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:42.119405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:42.259624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:42.268293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:42.269759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:42.270921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b43195e2-7969-49ae-b1bc-ccda27548f7e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.167874 Sep 27 04:22:42.271478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 161/334] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:42 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:42.470074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:42.470682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:42.477157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:42.477681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:42.478115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:42.478834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:42.479288 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:42.479711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:42.625894 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:42.633453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b7754466-5f0d-4eb1-bde9-281c1f970883 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.168870 Sep 27 04:22:42.637913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 174/335] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:42 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:43.296516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:43.297282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:43.303467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:43.303949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:43.304387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:43.305060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:43.305513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:43.305964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:43.659474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:43.660097 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:43.666890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:43.667374 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:43.667861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:43.668551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:43.669002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:43.669423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:43.686586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:43.696178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:43.697625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:43.698830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8d8ceef3-4a84-4ebd-8962-13d8ec36b8f4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.406229 Sep 27 04:22:43.699858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 162/336] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:43 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:43.813517 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:43.821584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-444971d2-626d-4a73-bf2c-bcaf62a687e0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.166303 Sep 27 04:22:43.822192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 175/337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:43 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:44.726304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:44.727011 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:44.733295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:44.733826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:44.734247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:44.734921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:44.735370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:44.735789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:44.844340 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:44.844981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:44.851606 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:44.852143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:44.852585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:44.853293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:44.853794 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:44.854197 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:44.872600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:44.881885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:44.883625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:44.885091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0032e156-9d2c-4504-b36a-b071caf89b11 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.162702 Sep 27 04:22:44.898197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 163/338] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:44 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:44.990319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:44.998273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d268aa1e-04c5-4ac2-bc97-9f9d35a95974 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.157698 Sep 27 04:22:44.999093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 176/339] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:44 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:45.912725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:45.913410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:45.921143 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:45.921708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:45.922172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:45.923811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:45.924590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:45.925171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:46.022291 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:46.022881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:46.032602 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:46.033101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:46.033546 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:46.037703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:46.038223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:46.038666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:46.075571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:46.084459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:46.085948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:46.087125 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ddc05567-d574-41fc-be87-0617f84c79b6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.178764 Sep 27 04:22:46.087679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 164/340] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:45 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:46.180939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:46.188486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-738022cd-719d-41f9-82c8-5268958d8824 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.170060 Sep 27 04:22:46.189689 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 177/341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:46 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:47.112320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:47.112936 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:47.119448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:47.119941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:47.120380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:47.121077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:47.121542 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:47.121991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:47.212152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:47.213071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:47.222216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:47.223089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:47.223851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:47.224885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:47.225713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:47.226389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:47.263179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:47.272973 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:47.274513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:47.275790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ef925a3f-685b-464a-9838-eef60a694cfc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.167552 Sep 27 04:22:47.276997 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 165/342] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:47 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:47.381125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:47.388612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9c4c1313-41f0-4851-a1d8-bc397133742a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.180343 Sep 27 04:22:47.389169 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 178/343] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:47 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:48.301820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:48.302434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:48.308779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:48.309270 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:48.309711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:48.310425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:48.310886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:48.311316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:48.412105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:48.412780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:48.420714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:48.422609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:48.423753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:48.424885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:48.425381 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:48.426086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:48.459878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:48.475680 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:48.477202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:48.478422 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-343275dc-5ac5-4e49-8db6-06fe401d974e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.180544 Sep 27 04:22:48.479176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 166/344] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:48 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:48.572018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:48.579645 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d8cddb81-d00d-4cf6-a75b-28ea764f0d03 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.171747 Sep 27 04:22:48.580210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 179/345] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:48 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:49.504677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:49.505357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:49.511759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:49.512248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:49.512688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:49.513378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:49.513867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:49.514297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:49.604461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:49.605081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:49.612789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:49.613355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:49.613812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:49.614506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:49.614959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:49.615410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:49.645529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:49.655869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:49.658169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:49.659859 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fb022bdf-0b31-49ef-a128-7a78d18fc73a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.159511 Sep 27 04:22:49.670209 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 167/346] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:49 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:49.746505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:49.754653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-260254f5-7030-4132-af3d-bd4f2cc1ae1e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.155752 Sep 27 04:22:49.755301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 180/347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:49 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:50.686014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:50.686713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:50.694135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:50.694629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:50.695071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:50.695767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:50.696225 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:50.696652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:50.780057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:50.781594 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:50.789242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:50.790143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:50.790805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:50.791749 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:50.792537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:50.793176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:50.849617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:50.853186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:50.860398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:50.862643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:50.863924 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8e333d80-7671-4f3a-b38f-0bdc3f96af98 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.181792 Sep 27 04:22:50.864489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 168/348] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:50 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:50.952874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:50.961242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4158c36f-c9ce-49ae-8771-58532103d185 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.185059 Sep 27 04:22:50.961869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 181/349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:50 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:51.892427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:51.893039 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:51.901405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:51.901933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:51.902370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:51.903069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:51.903523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:51.903942 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:51.953813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0f920c52-0548-4370-8c64-42567c1eef5e", "device": "/dev/sdb"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:51.959632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:51.967845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:51.976852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:51.977676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:51.980618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:51.984599 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:51.985302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:52.044986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:52.053792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:52.055611 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:52.057677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-aa2e368d-95b4-44d9-bea6-5eb02ad1be6c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.170690 Sep 27 04:22:52.071329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 169/350] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:51 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:52.085493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:52.086144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:52.092184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:52.092688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:52.093116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:52.093836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:52.094299 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:52.094724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:52.244814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:52.254155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-906a4fd5-a4a8-4df2-84d4-9ce47cc3f018 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.181352 Sep 27 04:22:52.257777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 170/351] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:52 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:53.088534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:53.089172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:53.095966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:53.096449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:53.096883 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:53.097566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:53.098054 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:53.098484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:53.239654 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:53.248748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:53.250214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:53.251424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8658904e-3fc9-4486-a667-d34db3b9bc22 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.166870 Sep 27 04:22:53.251965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 171/352] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:53 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:22:53.278309 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:53.278917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:53.285587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:53.286106 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:53.286547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:53.287236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:53.287693 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:53.288116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:53.428687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:53.436787 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6d22f137-a5cb-4f00-8a50-647fe40362c2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.161964 Sep 27 04:22:53.437406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 172/353] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:53 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:53.451160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ae6b7a4e-3506-4a7b-9b94-1eb06e76b1dc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.501282 Sep 27 04:22:53.451789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 182/354] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:22:51 2022] POST /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d/os-volume_attachments => generated 194 bytes in 1503 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:22:54.279876 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:54.280425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:54.287035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:54.287529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:54.287972 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:54.288670 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:54.289132 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:54.289563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:54.427233 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:54.435700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:54.437121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:54.438534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-456aea5b-6a09-497c-97a4-b3507b1a5d27 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.163348 Sep 27 04:22:54.439698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 173/355] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:54 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:54.461979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:54.462677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:54.468800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:54.469284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:54.469724 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:54.470935 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:54.471389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:54.471811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:54.486879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:22:54.611443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:54.619511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-70538a20-0f3d-4a98-81d7-790a345bdbfe tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.162278 Sep 27 04:22:54.620082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 183/356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:54 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:55.465490 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:55.466127 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:55.473343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:55.473870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:55.474338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:55.475655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:55.476113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:55.476637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:55.622763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:55.632766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:55.634220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:55.635380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b9f651d7-0e42-4243-a78a-5eaecf590c63 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.173932 Sep 27 04:22:55.636015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 174/357] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:55 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:55.644110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:55.645858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:55.652264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:55.652969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:55.653477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:55.654350 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:55.654870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:55.655367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:55.784776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:55.792338 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-56ea3e11-aa25-41e4-b658-023eb8a44b14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.152448 Sep 27 04:22:55.792898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 184/358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:55 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:56.663605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:56.664363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:56.671880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:56.672624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:56.673263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:56.674162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:56.674897 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:56.675468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:56.816635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:56.817405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:56.823961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:56.824453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:56.824898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:56.825903 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:56.826348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:56.827034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:56.840687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:56.851871 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:56.854245 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:56.856970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-20d4c121-9dac-469a-8219-f865358bce3c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.198622 Sep 27 04:22:56.857963 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 175/359] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:56 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:56.993351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:57.002298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1b880bce-4eff-44a2-a994-031addd3560b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.189279 Sep 27 04:22:57.013767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 185/360] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:56 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:22:57.883211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:57.883814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:57.890147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:57.890624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:57.891058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:57.891732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:57.892180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:57.892597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.018033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:58.026523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:58.026698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:58.027155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:58.027950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:58.029104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-114bccab-525f-4f09-b6c5-5516ea0b3de0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.149740 Sep 27 04:22:58.029652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 176/361] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:57 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:58.034442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.034919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.035348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.036028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.036479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.036900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.173062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:58.180795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b27d87fd-c55c-4007-a928-4ab240f3ebdd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1500 microversion: 2.1 time: 0.158047 Sep 27 04:22:58.181347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 186/362] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:58 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1500 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:58.203851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3ea786a4-428f-4af5-afee-ba0be4046c97"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:58.206861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:58.212696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.213176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.213609 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.214313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.214766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.215188 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.237730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] HTTP exception thrown: Cannot 'attach_volume' instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 while it is in vm_state rescued Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Request method failure captured: Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/os-volume_attachments HTTP/1.1 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 74 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMno35C93UkxtOc6ZDX0FxmutNBu7fDaa-k2OG5GytQSAWRNNp4ten4pQn4RNoqEXm6vU4l2Pd-YXND1zBwqYnB2amEJy47FN9XvUHmMJF5xEqgAj58Sq_crQCMS5k-CWu6CeJD8FcZr32ZbDrVLzgLbfnG2CFqm0KMqdAFrML2D_IHw Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:22:58.239906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c25f11f92d4e4fd8965fe630ad77f3c3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 2d0ba2ae3ffe4bb9b99f1347719df411 Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {"volumeAttachment": {"volumeId": "3ea786a4-428f-4af5-afee-ba0be4046c97"}} Sep 27 04:22:58.241503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Cannot 'attach_volume' instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 while it is in vm_state rescued Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'server_id': 'e5e47978-60aa-4191-b5c0-f6421a4e7e97', 'body': {'volumeAttachment': {'volumeId': '3ea786a4-428f-4af5-afee-ba0be4046c97'}}} Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:22:58.242488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 while it is in vm_state rescued Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:22:58.243892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 while it is in vm_state rescued Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Returning 409 to user: Cannot 'attach_volume' instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 while it is in vm_state rescued {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d4486ad8-d3ba-4216-b402-ff2cca1b3c66 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.041782 Sep 27 04:22:58.245180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 177/363] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:22:58 2022] POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/os-volume_attachments => generated 152 bytes in 43 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 04:22:58.254589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:22:58.255124 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:58.261065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.261547 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.262033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.262719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.263175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.263598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.327485 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-605885ce-f720-430b-9bd9-00c8a9ec326c tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/action" status: 202 len: 0 microversion: 2.1 time: 0.077050 Sep 27 04:22:58.328031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 187/364] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:22:58 2022] POST /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97/action => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:22:58.336709 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:58.337340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:58.343247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.343762 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.344197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.344872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:58.345939 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:58.346359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:58.483455 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:58.491832 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-34566d44-329b-441e-8081-3b70dbd8e8dc tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1508 microversion: 2.1 time: 0.158939 Sep 27 04:22:58.492398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 178/365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:58 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1508 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:22:59.056113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:59.056761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:59.062944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:59.063417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:59.064361 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:59.065045 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:59.065502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:59.065974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:59.194098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:59.202638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:59.204086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:59.205862 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-37218bc2-0e9c-4194-bec4-64772e4ba910 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.154589 Sep 27 04:22:59.206412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 188/366] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:22:59 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:22:59.514450 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:22:59.515067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:22:59.521439 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:59.521965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:59.522412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:59.523102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:22:59.523560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:22:59.523991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:22:59.651531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:22:59.660364 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c1ff2c24-d171-41d6-857b-fe257a8be6f8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1508 microversion: 2.1 time: 0.149772 Sep 27 04:22:59.660861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 179/367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:22:59 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1508 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:00.231468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:00.232071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:00.239087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:00.239567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:00.240004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:00.240693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:00.241148 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:00.241570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:00.370130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:00.379142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:00.380559 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:00.381695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-100394b9-5d03-4716-808f-b4aa2f3d7ec4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.154166 Sep 27 04:23:00.382520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 189/368] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:00 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:00.683110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:00.683631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:00.690211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:00.690700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:00.691137 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:00.691817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:00.692275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:00.692702 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:00.831083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:00.839510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c0a23e0c-08d0-4791-82d7-5b01550a6665 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1508 microversion: 2.1 time: 0.160364 Sep 27 04:23:00.840066 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 180/369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:00 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1508 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:01.406902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:01.408739 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:01.415531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:01.416017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:01.416460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:01.417152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:01.417601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:01.418049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:01.547303 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:01.556385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:01.557858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:01.559016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ae0e4340-212d-4c25-a8b2-6fac1a81b59b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.156043 Sep 27 04:23:01.559571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 190/370] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:01 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:01.862983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:01.863495 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:01.869801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:01.870285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:01.870716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:01.871392 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:01.871841 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:01.872259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:02.000740 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:02.008416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-295e21c7-45d4-4131-9d31-b00adca05c82 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1514 microversion: 2.1 time: 0.150051 Sep 27 04:23:02.008969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 181/371] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:01 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:02.584030 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:02.584635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:02.591634 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:02.592115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:02.592551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:02.593236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:02.593689 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:02.594140 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:02.728237 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:02.737703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:02.739408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:02.740745 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-93aecd74-44bb-4ace-a7ed-500d275a0a41 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.161117 Sep 27 04:23:02.741448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 191/372] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:02 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:03.346547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-14aa2bca-b4ba-4a3b-860c-dd25c14841cf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: .version_select at 0x7fce70c025e0>, body: {"keypair": {"name": "tempest-keypair-800992897", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE33sFsNyFCRR6euHm3/VPXYf58+qtFdCoBklhsqNvsvm7SQp0NY2RTwo8kTmDpz7uZIv90oAFaiYorL+WtYi5k0KsqVzP031jDnhJnHeWcZHzWN50vHOaU7Kpv4Z4A/LQ=="}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:03.366769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-14aa2bca-b4ba-4a3b-860c-dd25c14841cf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:23:03.370463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-14aa2bca-b4ba-4a3b-860c-dd25c14841cf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for user 2d0ba2ae3ffe4bb9b99f1347719df411 and project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:23:03.405461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-14aa2bca-b4ba-4a3b-860c-dd25c14841cf tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.062824 Sep 27 04:23:03.406090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 182/373] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:23:03 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:03.766487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:03.767215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:03.769757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=52 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:03.774898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:03.775392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:03.775824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:03.776610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:03.777033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:03.777523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:03.927702 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:03.936440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:03.938016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:03.939256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-350ac0c9-ece4-4e8a-8c37-1a1376799824 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.176668 Sep 27 04:23:03.939898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 192/374] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:03 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:04.709880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-783676267", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3b03541e-a362-4f1f-858b-b7051275090d"}], "security_groups": [{"name": "tempest-securitygroup--808183741"}], "key_name": "tempest-keypair-800992897", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:04.817297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] validate_networks() for [('3b03541e-a362-4f1f-858b-b7051275090d', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:23:04.966493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:04.967090 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:04.973951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:04.974436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:04.974871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:04.975572 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:04.976012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:04.976435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:05.106612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:05.116175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:05.117631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:05.118869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8f03ad32-2551-4601-b21e-3774e4695be7 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.156317 Sep 27 04:23:05.119605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 193/375] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:04 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:23:05.190317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:23:05.190742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:23:05.191218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:23:05.191630 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:23:05.192046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:23:05.200572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:05.201051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:05.201485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:05.202254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:05.202709 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:05.203133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:05.219619 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:23:05.223940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Getting quotas for user 2d0ba2ae3ffe4bb9b99f1347719df411 and project c25f11f92d4e4fd8965fe630ad77f3c3. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:23:05.234628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:23:05.236881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:23:05.282712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b4478e38-fcd4-4a2d-ac63-a84bef367071 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.577045 Sep 27 04:23:05.283454 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 183/376] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:23:04 2022] POST /compute/v2.1/servers => generated 408 bytes in 579 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:23:05.294911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a5d1ba78-4270-4a5e-9090-b3ce044a201d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:05.295589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a5d1ba78-4270-4a5e-9090-b3ce044a201d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:05.349401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a5d1ba78-4270-4a5e-9090-b3ce044a201d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:05.356590 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a5d1ba78-4270-4a5e-9090-b3ce044a201d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1244 microversion: 2.1 time: 0.065453 Sep 27 04:23:05.357344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 194/377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:05 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1244 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:06.144548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:06.145148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:06.151170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:06.151656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:06.152086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:06.152777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:06.153211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:06.153627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:06.279891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:06.288524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:06.289985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:06.291130 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-47e51511-b4ad-440a-88c7-8af2451af000 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.151007 Sep 27 04:23:06.291678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 184/378] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:06 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:06.379698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:06.380303 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:06.387358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:06.387838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:06.388274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:06.388958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:06.389409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:06.389859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:06.448871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:06.456661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7da4ad4a-2652-49ef-9adb-bca0c4430ec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1248 microversion: 2.1 time: 0.081394 Sep 27 04:23:06.457223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 195/379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:06 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1248 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:07.318372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:07.318990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:07.325148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:07.325636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:07.326420 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:07.326895 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:07.327346 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:07.328048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:07.473226 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:07.479889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:07.481545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:07.484067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:07.485502 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:07.486711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-42b609cf-b85f-42ce-8502-76ee94fd9ead tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.173176 Sep 27 04:23:07.487289 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 185/380] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:07 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:07.489656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:07.490194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:07.490636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:07.491341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:07.491802 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:07.492232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:07.590405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:07.599003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-33768beb-8469-4d5d-a5f1-2e6137c7b2c0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1240 microversion: 2.1 time: 0.123015 Sep 27 04:23:07.599447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 196/381] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:07 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1240 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:08.186918 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "name": "network-vif-plugged", "status": "completed", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:08.198187 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:08.199947 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:08.200472 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:08.215245 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] Creating event network-vif-plugged:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:23:08.220403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-9985ec3f-e579-4e9e-9e83-285eabaceb01 req-f216562b-1cf0-47b5-931a-d5e0fa8ee3d5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037241 Sep 27 04:23:08.221586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 186/382] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:08.512434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:08.513032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:08.521264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:08.521769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:08.522202 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:08.522889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:08.523344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:08.523763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:08.625102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:08.625920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:08.633659 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:08.634198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:08.635015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:08.636031 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:08.637176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:08.637805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:08.662971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:08.674344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:08.676620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:08.677880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-39150bcc-46b4-493a-9c89-afd6097f2ace tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.169370 Sep 27 04:23:08.678846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 197/383] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:08 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:08.716395 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:08.726855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-edf838c1-66ba-40bb-9922-f98641aebec9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1296 microversion: 2.1 time: 0.107230 Sep 27 04:23:08.736235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 187/384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:08 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1296 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:09.704752 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:09.705282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:09.711653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:09.712125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:09.712553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:09.713231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:09.713682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:09.714124 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:09.750343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:09.751067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:09.757810 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:09.758345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:09.758695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:09.759385 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:09.759843 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:09.760265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:09.765860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:09.824880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:09.834618 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-be8c9561-bd8b-4e71-b87e-b5f309a7440b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1296 microversion: 2.1 time: 0.087967 Sep 27 04:23:09.835193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 188/385] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:09 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1296 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:09.851653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:09.860103 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:09.861525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:09.862814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-72ff8e14-d198-4e03-b99f-60f2efa25592 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.162059 Sep 27 04:23:09.863358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 198/386] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:09 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:10.857613 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:10.858262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:10.864564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:10.865052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:10.865493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:10.866394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:10.866779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:10.867211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:10.888216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:10.889025 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:10.897002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:10.897852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:10.898361 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:10.899301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:10.899766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:10.900191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:10.944875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:10.952522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-93e32ea6-0467-469a-9890-2b19f4587e72 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1296 microversion: 2.1 time: 0.098735 Sep 27 04:23:10.953085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 189/387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:10 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1296 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:11.029042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:11.037871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:11.039257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:11.040405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-43d61e67-3caa-4976-aef5-a2267cebdf6e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.156604 Sep 27 04:23:11.040962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 199/388] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:10 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:23:11.976780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:11.977463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:11.984871 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:11.986014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:11.986463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:11.988049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:11.988514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:11.988948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:12.065151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:12.065767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:12.066439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:12.073298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:12.073453 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8e8bc88b-8079-4cf0-8286-3a3d049be209 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1302 microversion: 2.1 time: 0.100670 Sep 27 04:23:12.074110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:12.074235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 190/389] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:11 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1302 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:12.081644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:12.082434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:12.082901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:12.083357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:12.087810 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:12.216067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:12.225021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:12.226496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:12.227627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7ef04f10-2b88-415e-9112-be4a91b97c62 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.166051 Sep 27 04:23:12.228171 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 200/390] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:12 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:23:13.096604 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:13.097543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:13.099872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=4 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:13.105079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:13.105575 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:13.106214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:13.106924 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:13.107390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:13.107822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:13.237724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:13.245299 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b735f1cb-bd4c-4740-8fb2-ac450d755765 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.152691 Sep 27 04:23:13.246082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 191/391] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:13 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:13.252368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:13.253060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:13.259602 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:13.260085 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:13.260517 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:13.261194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:13.261648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:13.262092 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:13.500404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:13.510044 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:13.511588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:13.512751 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-895970a1-0814-4098-901f-b593a8b7bf94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.264354 Sep 27 04:23:13.513322 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 201/392] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:13 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:13.933610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:13.943450 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:13.943935 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:13.944373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:13.954829 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] Creating event network-changed:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:23:13.960391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-7cc1793a-4b06-4b3f-a969-aa38e3045657 req-3e3c3858-c40c-4105-97e0-caf83a5f6c72 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030521 Sep 27 04:23:13.960986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 192/393] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:23:14.267686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:14.268881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:14.275276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:14.275768 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:14.276208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:14.276904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:14.277363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:14.277817 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:14.405960 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:14.414079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1af6ac11-ae19-49cd-aefa-e1aef73c83a2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.150250 Sep 27 04:23:14.414632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 202/394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:14 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:14.540449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:14.541151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:14.547299 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:14.547788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:14.548233 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:14.548950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:14.549417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:14.549881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:14.677405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:14.685905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:14.687355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:14.688522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-331718fc-43f1-412f-9707-ed6812f53688 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.152787 Sep 27 04:23:14.689076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 193/395] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:14 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:23:15.215589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:15.217881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:15.223761 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.224242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.224700 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.225379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.225858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.226278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.298584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9f336a09-903f-48c4-ac35-6e406f73fa07 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.086693 Sep 27 04:23:15.299210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 203/396] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:23:15 2022] POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:23:15.307684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:15.308354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:15.310477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:15.311133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=10 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:15.317345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.317861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.318310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.318999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.319460 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.319888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.433689 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:15.437971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:15.438660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:15.445641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.446140 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.446578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.447260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.447712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.448133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.477251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:15.486655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a15bd035-3c99-43b7-9474-437ce75311c8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1753 microversion: 2.1 time: 0.182758 Sep 27 04:23:15.487833 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 194/397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:15 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1753 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:15.574629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:15.582195 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1315aab3-477c-45eb-a004-176629089e89 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.149417 Sep 27 04:23:15.582753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 204/398] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:15 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:15.715235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:15.715881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:15.722314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.722799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.723234 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.723912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:15.724366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:15.724790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:15.849922 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:15.858377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:15.859817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:15.860969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b8cce387-4412-4018-a5ff-e08d2450606a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.149647 Sep 27 04:23:15.861525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 195/399] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:15 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:16.511728 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:16.512300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:16.518678 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.519809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.520242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:16.520924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.521371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.521818 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:16.604600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:16.605290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:16.611616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.612109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.613282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:16.613806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.614293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.614882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:16.684552 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:16.695411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-817f3277-35c4-4308-88fe-2a8785d40616 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1753 microversion: 2.1 time: 0.187515 Sep 27 04:23:16.696570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 205/400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:16 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1753 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:16.759363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:16.768037 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9fe15bca-71c3-4f7c-8f24-9910cf7546eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.167345 Sep 27 04:23:16.768830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 196/401] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:16 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:16.885657 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:16.886292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:16.892271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.892753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.893192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:16.893908 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:16.894396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:16.894823 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:17.076296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:17.086739 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:17.088280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:17.089720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-46f14057-fb29-43f3-94cc-32ba4c63afdc tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.207941 Sep 27 04:23:17.101766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 206/402] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:16 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:17.719287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:17.719903 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:17.726900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:17.727387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:17.727825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:17.728512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:17.728968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:17.729395 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:17.791580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:17.792098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:17.798473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:17.798958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:17.799396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:17.800084 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:17.801159 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:17.801589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:17.872861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:17.881239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2b8d4517-6e1e-4ede-ad7d-e0e8488d800f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.165851 Sep 27 04:23:17.881824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 197/403] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:17 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:17.928541 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:17.936066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-caae34fe-eb31-4676-9df4-ecf8ad090bd7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.148439 Sep 27 04:23:17.936618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 207/404] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:17 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:18.116172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:18.116737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:18.122570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.123053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.123520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:18.124194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.124653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.125075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:18.251259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:18.259742 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:18.261169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:18.262341 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4b074321-f31b-457b-9eca-8d6ac3e8bca2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1808 microversion: 2.20 time: 0.150103 Sep 27 04:23:18.262890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 198/405] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:18 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1808 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:23:18.904699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:18.905310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:18.911946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.912418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.912846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:18.913520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.914002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.914422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:18.958585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:18.959192 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:18.965543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.966082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.966520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:18.967198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:18.967650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:18.968075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:19.062301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:19.073566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-99face9b-90ba-4615-9f98-10059f278de3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.172595 Sep 27 04:23:19.074168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 208/406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:18 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:19.121181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:19.129371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1cad24b3-6603-4e4e-99cd-46049657dc0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.174003 Sep 27 04:23:19.129958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 199/407] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:18 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:19.288795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:19.289408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:19.295335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:19.295825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:19.296262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:19.296956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:19.297416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:19.298081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:19.429205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:19.437915 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:19.439349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:19.440505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a6412766-8df1-49dd-9b8b-203e5f440939 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1795 microversion: 2.20 time: 0.156096 Sep 27 04:23:19.441059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 209/408] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:23:19 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1795 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:23:20.097042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:20.097690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:20.104117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.104632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.105092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.105775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.106239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.106669 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.151899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:20.152507 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:20.159081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.159553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.159991 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.160677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.161134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.161555 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.245555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:20.254610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ffd8f416-17d8-46d9-bab8-91ef3c9c000b tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.161389 Sep 27 04:23:20.255186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 200/409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:20 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:20.290864 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:20.298633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b299c378-01a2-4ec2-b2e5-5155deb91ec4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.150532 Sep 27 04:23:20.299269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 210/410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:20 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:20.541081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:20.541577 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:20.547465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.547952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.548390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.549070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:20.549530 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:20.549989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:20.676749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lazy-loading 'services' on Instance uuid 7ff20f8a-f72b-4c87-8c62-dbdf2714536d {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:23:20.786277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1761ba0d-d5d5-4aa1-8669-224a80f1d6b4 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d/os-volume_attachments/0f920c52-0548-4370-8c64-42567c1eef5e" status: 202 len: 0 microversion: 2.1 time: 0.248919 Sep 27 04:23:20.786870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 201/411] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:23:20 2022] DELETE /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d/os-volume_attachments/0f920c52-0548-4370-8c64-42567c1eef5e => generated 0 bytes in 250 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:23:21.279253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:21.280001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:21.287524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.288010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.288449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.289138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.289595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.290079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.323541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:21.325954 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:21.348227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.348867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.349341 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.350083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.350546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.350977 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.456087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:21.466727 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-53d64601-d999-4b88-b56f-3145039da490 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.190092 Sep 27 04:23:21.469782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 211/412] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:21 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:21.514982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:21.523758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e114fe55-deec-4cb7-83c2-9aa27776b5ff tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.205295 Sep 27 04:23:21.524335 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 202/413] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:21 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:21.915282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:21.915825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:21.921632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.922144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.922579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.923256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:21.923706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:21.924129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:21.948721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:23:22.028871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-38b92f6b-589f-4725-9101-0d22bd065431 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 204 len: 0 microversion: 2.1 time: 0.116948 Sep 27 04:23:22.029479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 212/414] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:23:21 2022] DELETE /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:23:22.038040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:22.038660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:22.044686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.045431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.045901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.046614 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.047071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.047503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.181676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:22.189231 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1e5b9337-f568-4b2d-bfb7-c217e72e1635 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1514 microversion: 2.1 time: 0.155148 Sep 27 04:23:22.189808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 203/415] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:22 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:22.490135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:22.490831 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:22.497328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.497837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.498270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.498951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.499397 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.499813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.546547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:22.547163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:22.557766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.558281 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.558802 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.559565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:22.560134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:22.560531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:22.643889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:22.652493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8182e3f0-97bf-4fb8-b71c-8b1bd6dc2e71 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.166597 Sep 27 04:23:22.653144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 213/416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:22 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:22.691691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:22.695198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:22.700085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8af8a542-a80c-483c-9f55-06bd244c16aa tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.157407 Sep 27 04:23:22.700646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 204/417] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:22 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:23.212528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:23.213135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:23.218966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.219446 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.219881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.220555 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.221002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.221420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.226452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:23.355790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:23.363551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-affe07f0-991c-4ba8-a4a3-df8bcbbb2391 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1514 microversion: 2.1 time: 0.154923 Sep 27 04:23:23.364123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 214/418] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:23 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:23.385197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "name": "network-vif-plugged", "status": "completed", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:23.395543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.396025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.396456 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.412093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] Creating event network-vif-plugged:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:23:23.417013 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-e2b0c29e-6997-493c-a245-fd93e99f938b req-06b217fb-89bc-4d33-92be-fc4165eefd54 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035638 Sep 27 04:23:23.418069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 205/419] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:23.676223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:23.676821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:23.684082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.684558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.684986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.685666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.686144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.686559 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.725553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:23.726277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:23.732326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.732807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.733245 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.733961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:23.734416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:23.734838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:23.846393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:23.855119 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5ec2e468-c4a1-4224-b023-793f072e389a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.182703 Sep 27 04:23:23.856212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 215/420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:23 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:23.888280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:23.896475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5be50372-5d20-4272-a2fb-dd0c034d038e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.175412 Sep 27 04:23:23.897028 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 206/421] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:23 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:24.386862 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:24.387471 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:24.393444 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.393951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.394388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:24.395093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.395544 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.395968 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:24.749884 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:24.757498 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-48c8facd-93ea-4879-a594-05c2d98d7624 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1514 microversion: 2.1 time: 0.374522 Sep 27 04:23:24.758099 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 216/422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:24 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1514 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:24.879744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:24.880360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:24.887752 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.888282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.888891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:24.889600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.890084 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.890516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:24.919487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:24.920096 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:24.925960 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.926441 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.926876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:24.927559 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:24.928011 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:24.928430 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:25.059339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:25.060732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:25.068093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9e57478c-18a1-432e-bd7e-cb7bcf45d721 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.192171 Sep 27 04:23:25.068383 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ae75d390-daad-4cce-9d29-966e7144854b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.152904 Sep 27 04:23:25.068955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 217/423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:24 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:25.073302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ff20f8a-f72b-4c87-8c62-dbdf2714536d", "name": "network-vif-deleted", "tag": "44284c30-6edf-45a3-8b70-fc4bb36bc551"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:25.082047 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 207/424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:24 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:25.082918 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:25.083396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:25.083827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:25.094857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] Creating event network-vif-deleted:44284c30-6edf-45a3-8b70-fc4bb36bc551 for instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d on hv-cinder-85918 Sep 27 04:23:25.100147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-8c4e0c07-be08-46cb-8e93-84218cbbecb0 req-ace7e1f9-475b-4199-bf0a-c6c8a4e36c85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030114 Sep 27 04:23:25.100710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 218/425] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:25.782292 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:25.783164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:25.789284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:25.789806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:25.790274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:25.790967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:25.791426 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:25.791854 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:25.822933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:25.823409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:25.823844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:25.824289 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lazy-loading 'fault' on Instance uuid 7ff20f8a-f72b-4c87-8c62-dbdf2714536d {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:23:25.866591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:25.874053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b1814b9b-9447-4ebe-9987-e6d6a3a0f536 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 200 len: 1309 microversion: 2.1 time: 0.095567 Sep 27 04:23:25.874616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 208/426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:25 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 1309 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:26.092499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:26.093153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:26.093711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:26.094435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:26.100702 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.101322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.101788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.102482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.102943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.103368 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.109008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.109939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.109939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.110936 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.111380 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.112453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.246148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:26.257294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1dd76865-4ab6-48fb-b755-70413094abc2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1369 microversion: 2.1 time: 0.168720 Sep 27 04:23:26.257950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 209/427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:26 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1369 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:26.270669 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:26.278811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4a8afe53-fad5-4159-8c20-1b0786a6bd38 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.190200 Sep 27 04:23:26.279369 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 219/428] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:26 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:26.897561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:26.898202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: 7ff20f8a-f72b-4c87-8c62-dbdf2714536d] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:26.904086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.904565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.904999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.905672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:26.906147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:26.906568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:26.920185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] HTTP exception thrown: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Request method failure captured: Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d HTTP/1.1 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnpkzET_rz6RAox7lvhcBQwYD-2NVrhJbKJHRpCoo4tXt4pbbdnTbbVArHX-P_LcErSAjK67BhREHz9CsgdZRZJuO35MLpyhdegYUtRU8V3-ZsKENmqU18blhGZeagCgOScYuP5SWpZtnWUJBvOGXWa2CjbppTIehLslu3x6EVjW4qc Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 14d42cf78c32468cb8bf10aa847c7cc3 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:23:26.921801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_14d42cf78c32468cb8bf10aa847c7cc3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 14d42cf78c32468cb8bf10aa847c7cc3 Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesSnapshotTestJSON-699738051-project Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 782ebdc88e21483eb3cecd00415b9b07 Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesSnapshotTestJSON-699738051-project Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. Sep 27 04:23:26.923304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '7ff20f8a-f72b-4c87-8c62-dbdf2714536d'} Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:23:26.924121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. Sep 27 04:23:26.925616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:23:26.927548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Returning 404 to user: Instance 7ff20f8a-f72b-4c87-8c62-dbdf2714536d could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:23:26.927548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c6a75126-e475-4a59-b81a-75c8231b053e tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d" status: 404 len: 111 microversion: 2.1 time: 0.029472 Sep 27 04:23:26.927548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 210/429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:26 2022] GET /compute/v2.1/servers/7ff20f8a-f72b-4c87-8c62-dbdf2714536d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:23:27.280954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:27.281589 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:27.288901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.289409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.289876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.290567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.291024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.291472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.302147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:27.302887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:27.309313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.309845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.310291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.310982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.311442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.311870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.421244 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:27.428976 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-17cfb727-2a52-4320-b1b2-1964eee56d63 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1563 microversion: 2.1 time: 0.152144 Sep 27 04:23:27.429661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 220/430] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:27 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1563 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:27.451542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:27.452019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:27.454276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:27.458480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.458957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.459472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.460156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:27.460609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:27.461024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:27.463044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ee68323d-4d9f-4ed0-a2fd-7a4a787ed6d8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.164724 Sep 27 04:23:27.463892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 211/431] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:27 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:27.541674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a615f190-b1d5-4821-a69f-67d4b683badd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093720 Sep 27 04:23:27.542294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 221/432] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:23:27 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:23:28.454526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:28.465154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:28.465771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:28.466300 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:28.479298 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] Creating event network-changed:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:23:28.484487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-1a24e555-3ee0-4775-9955-bdab8bab1abc req-97f60320-3845-463e-881b-74026d8a70b0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033440 Sep 27 04:23:28.485469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 212/433] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:28.488585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:28.490501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:28.497193 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:28.497813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:28.498323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:28.499289 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:28.499823 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:28.500450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:28.656359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:28.665233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-878b8d10-bad5-4ffa-8d2f-1834bb9b143f tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.183180 Sep 27 04:23:28.665857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 222/434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:28 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:29.689334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:29.689996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:29.696068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:29.696554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:29.696999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:29.697684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:29.698166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:29.698593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:29.837393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:29.845364 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0675986d-d7ae-422f-a06b-6a85d6308304 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.160782 Sep 27 04:23:29.845944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 213/435] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:29 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:30.869661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:30.870302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:30.876125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:30.876606 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:30.877044 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:30.877734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:30.878207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:30.878634 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:31.018696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:31.030162 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-043a9f5f-2983-4900-9dd7-3e9aa08516d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.165022 Sep 27 04:23:31.030744 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 223/436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:30 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:32.053758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:32.054370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:32.060741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:32.061223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:32.061666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:32.062369 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:32.063221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:32.063660 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:32.205956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:32.214560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c38e7428-d520-495c-a907-5aa69ecdf7a1 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.164702 Sep 27 04:23:32.215123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 214/437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:32 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:33.237821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:33.238425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:33.244843 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:33.245322 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:33.245776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:33.246454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:33.246902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:33.247321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:33.389067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:33.397767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4babc849-548a-436d-b758-6447f363d56a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.163827 Sep 27 04:23:33.398329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 224/438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:33 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:34.421119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:34.421734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:34.428446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:34.428932 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:34.429370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:34.430092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:34.430553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:34.430984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:34.570631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:34.580449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6cfc9ba0-0cd3-4851-bc4f-4d1789bff511 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.163173 Sep 27 04:23:34.581011 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 215/439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:34 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:35.604291 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:35.604992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:35.607460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:35.612774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:35.613269 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:35.614295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:35.614993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:35.615454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:35.615881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:35.756686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:35.764728 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1ba28479-5f59-4a5d-aeee-5b848deff8d3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.164264 Sep 27 04:23:35.765282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 225/440] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:35 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:36.788664 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:36.789563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:36.795998 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:36.796492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:36.796928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:36.797608 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:36.798082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:36.798505 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:36.812980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1827894921", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "networks": [{"uuid": "14e40f69-7e37-43ea-ba44-fd89b3b60419"}]}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:36.868310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] validate_networks() for [('14e40f69-7e37-43ea-ba44-fd89b3b60419', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:23:36.943590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:36.951622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e83e31bd-d3a3-461a-bf97-54c1be5250f8 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.167563 Sep 27 04:23:36.952186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 216/441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:36 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:37.350874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:23:37.351295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:23:37.351766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:23:37.352171 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:23:37.352603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:23:37.361161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:37.361634 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:37.362117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:37.362827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:37.363275 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:37.364316 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:37.380666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Getting quotas for project 14d42cf78c32468cb8bf10aa847c7cc3. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:23:37.384518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Getting quotas for user 782ebdc88e21483eb3cecd00415b9b07 and project 14d42cf78c32468cb8bf10aa847c7cc3. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:23:37.395087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:23:37.397422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:23:37.444142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-11753184-8b31-422c-bc88-e5ff01a61f88 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.635062 Sep 27 04:23:37.444783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 226/442] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:23:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 636 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 04:23:37.455676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5dcc0d9b-00f2-4cc2-a6f5-623a6f248f17 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:37.456388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5dcc0d9b-00f2-4cc2-a6f5-623a6f248f17 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:37.513107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5dcc0d9b-00f2-4cc2-a6f5-623a6f248f17 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:37.520358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5dcc0d9b-00f2-4cc2-a6f5-623a6f248f17 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1219 microversion: 2.1 time: 0.068045 Sep 27 04:23:37.521052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 217/443] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:37 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1219 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:37.975666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:37.976276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:37.982453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:37.982930 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:37.983363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:37.984038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:37.984488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:37.984925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:38.149304 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:38.157553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ac106c8d-fb83-4561-a0ac-ac47f3370b13 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.186626 Sep 27 04:23:38.158420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 227/444] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:37 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:38.543225 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:38.543836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:38.550255 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:38.550682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:38.551620 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:38.552301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:38.552757 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:38.553182 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:38.613028 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:38.621010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a60f1936-29b9-4b10-a06a-7aff456b92e3 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1271 microversion: 2.1 time: 0.081626 Sep 27 04:23:38.621576 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 218/445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:38 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1271 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:39.110608 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:39.111149 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:39.117520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.118036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.118472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.119160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.119612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.120034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.190906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:39.191879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:39.199239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.200103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.200792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.201499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.202145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.202566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.258039 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'services' on Instance uuid e24dba8d-ecd7-482f-9c93-79e714785fec {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:23:39.375872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-203aba3f-88af-46c9-a29c-bb5db10f9a03 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments/7207a7df-705c-49f2-b0e0-6eec0c33de7d" status: 202 len: 0 microversion: 2.1 time: 0.269043 Sep 27 04:23:39.376785 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 228/446] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:23:39 2022] DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments/7207a7df-705c-49f2-b0e0-6eec0c33de7d => generated 0 bytes in 271 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:23:39.471046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:39.481181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2aa5f4bc-e7ac-4a1e-b1a7-9977b8a2000e tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.294497 Sep 27 04:23:39.482307 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 219/447] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:39 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:39.645536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:39.646266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:39.652408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.652888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.653323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.654038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:39.654495 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:39.654921 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:39.780859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:39.788710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bd7673d7-bfa4-4c4c-b4e2-be94b9fb58b7 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.148045 Sep 27 04:23:39.789272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 229/448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:39 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:40.509066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc919c44-b977-44e9-9c89-e92bcf1cebe6", "tag": "0fe69958-0631-4361-8934-1da42bed69db"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:40.513216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:40.518154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:40.536826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.537319 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.537785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.542614 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.543102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.543531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.544844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.545492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.545953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.563100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] Creating event network-changed:0fe69958-0631-4361-8934-1da42bed69db for instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 on hv-cinder-85918 Sep 27 04:23:40.569737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-661d07ca-0b35-46af-bae1-e24dff21a5ab req-eb45639d-e956-4392-8534-5b6fe368f43f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067039 Sep 27 04:23:40.571579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 230/449] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:40.579240 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:40.590342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.590974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.591490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.652396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Going to try to stop instance {{(pid=120423) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 04:23:40.677058 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:40.697775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e85589e0-07b8-4e7d-93f6-b34380b20aee tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action" status: 202 len: 0 microversion: 2.1 time: 0.124711 Sep 27 04:23:40.698396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 231/450] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:23:40 2022] POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:23:40.706867 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:40.707607 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:40.713796 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.714522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.715129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.716011 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.716617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.717194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.774318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:40.784406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-271e07dc-0ab0-42f1-abd5-f2e5be41e3bf tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.283084 Sep 27 04:23:40.785166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 220/451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:40 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:40.811286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:40.812052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:40.819527 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.820280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.820852 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.821679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:40.822257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:40.822981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:40.848158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:40.855949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-20c3d3f9-a100-48d1-9c53-0b3f858c5d8a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.153160 Sep 27 04:23:40.856516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 232/452] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:40 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:40.954734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:40.962261 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-983c4ebb-b704-46a6-ba0b-e3896211ad50 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.155096 Sep 27 04:23:40.962820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 221/453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:40 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:41.808157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:41.808759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:41.815073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.815548 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.815979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:41.817155 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.817711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.818159 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:41.880353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:41.880872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:41.888507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.889008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.889458 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:41.891694 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.894070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.894520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:41.964602 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:41.972617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7910ddb4-77e5-4624-a1a8-bcf9557218e4 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1758 microversion: 2.1 time: 0.168352 Sep 27 04:23:41.973352 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 233/454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:41 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1758 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:41.984726 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:41.985476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:41.991279 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.991758 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.992196 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:41.992885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:41.993345 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:41.993790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:42.025007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:42.032838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-93c4e433-638b-4f76-83a7-55e3d128f796 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.156984 Sep 27 04:23:42.033668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 222/455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:41 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:42.120354 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:42.127896 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f2b72de1-2858-41b5-a7d5-460d6100b50d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.146817 Sep 27 04:23:42.128452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 234/456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:41 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:42.995783 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:42.996390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:43.002972 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.003451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.003884 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.005155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.005605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.006061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.057376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:43.058008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:43.064347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.064816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.065248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.065958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.066409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.067505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.150329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:43.158758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2a2a1d25-ff4d-44a3-930a-ed3d22f12c86 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1761 microversion: 2.1 time: 0.166530 Sep 27 04:23:43.159321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 223/457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:42 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1761 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:43.164431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:43.165035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:43.168325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=70 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:43.172606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.173091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.173701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.174551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:43.175003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:43.175612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:43.181593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=100 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:43.210831 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:43.218824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e22e27b1-0a6e-4a1f-aef6-b632506dc163 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.165655 Sep 27 04:23:43.219392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 235/458] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:43 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:43.320082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:43.328409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e745fcd5-edc7-4318-98db-38a305a634ea tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.168111 Sep 27 04:23:43.329181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 224/459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:43 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:44.244031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:44.244686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:44.251130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:44.251615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:44.252058 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:44.252749 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:44.253210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:44.253639 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:44.351351 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:44.351959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:44.358407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:44.358905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:44.359327 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:44.360016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:44.360472 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:44.360896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:44.381182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:44.388870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6960a268-c9cc-4ab0-9203-0175cdf82603 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.149247 Sep 27 04:23:44.389438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 236/460] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:44 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:44.492248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:44.499880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-440ca3d4-8cee-456b-8334-bc11702cf4f0 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.152338 Sep 27 04:23:44.500373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 225/461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:44 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:45.412959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:45.413772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:45.420643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:45.421367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:45.422036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:45.422751 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:45.423213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:45.423643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:45.527384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:45.528180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:45.534952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:45.535506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:45.536092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:45.537682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:45.538221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:45.538713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:45.572966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:45.582180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c0a06f0a-58cb-48af-ae80-95f327858178 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.173829 Sep 27 04:23:45.582762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 237/462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:45 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:45.596277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc919c44-b977-44e9-9c89-e92bcf1cebe6", "name": "network-vif-plugged", "status": "completed", "tag": "0fe69958-0631-4361-8934-1da42bed69db"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:45.606775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:45.607410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:45.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:45.620210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] Creating event network-vif-plugged:0fe69958-0631-4361-8934-1da42bed69db for instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 on hv-cinder-85918 Sep 27 04:23:45.625351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-ee97820b-1fc3-4ce3-a2e5-ed7ed63aa73c req-7476214b-b1df-40ba-8c2c-8a4f34a4b9dc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032794 Sep 27 04:23:45.626042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 238/463] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:23:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:45.689855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:45.697570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-07b4f82d-be5a-4d4e-9287-4450201dd9fc tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.173941 Sep 27 04:23:45.698316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 226/464] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:45 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:46.605366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:46.606100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:46.612185 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:46.612662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:46.613094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:46.613795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:46.614249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:46.615199 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:46.720313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:46.720926 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:46.727667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:46.728156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:46.728602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:46.729326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:46.729805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:46.730234 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:46.742184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:46.749910 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9a42b75f-51a9-4416-aad4-926874e27293 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.149172 Sep 27 04:23:46.750560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 239/465] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:46 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:46.856383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:46.863826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fef06bb9-9e86-4bea-a528-ebe761829e81 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1319 microversion: 2.1 time: 0.147356 Sep 27 04:23:46.864384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 227/466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:46 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:47.773635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:47.774279 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:47.777052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:47.778042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:47.784538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:47.785501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:47.786038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:47.786786 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:47.787239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:47.787691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:47.888005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:47.888617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:47.890526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:47.897060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:47.897555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:47.898018 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:47.898714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:47.899171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:47.899600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:47.921715 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:47.929419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2e37d02d-2cf2-4027-add7-6fa2f8340877 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.160603 Sep 27 04:23:47.930013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 240/467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:47 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:48.025086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:48.032695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c2f36053-d2ed-4056-81fa-740deeab657a tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1508 microversion: 2.1 time: 0.148465 Sep 27 04:23:48.033254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 228/468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:47 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1508 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:48.055588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e0a8a204-89bd-49f7-9254-950534df115c", "device": "/dev/sdb"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:48.057978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:48.064065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:48.064553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:48.064996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:48.065682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:48.066168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:48.066604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:48.296833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a7ec6122-8033-47bc-a5dc-18da07f7d042 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "POST /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.245443 Sep 27 04:23:48.297499 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 241/469] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:23:48 2022] POST /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6/os-volume_attachments => generated 194 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:23:48.952937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:48.953557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:48.960425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:48.960912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:48.961349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:48.962055 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:48.962512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:48.962938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:49.090288 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:49.098096 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8eb41100-48c4-4eb9-adbf-459312dc47db tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.148990 Sep 27 04:23:49.098651 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 229/470] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:48 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:50.122952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:50.123656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:50.129949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:50.130439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:50.130882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:50.131572 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:50.132032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:50.132461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:50.267483 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:50.275654 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9142a762-aa18-4c60-bdcc-afefeb411fdb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.157535 Sep 27 04:23:50.276230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 242/471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:50 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:51.300015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:51.300756 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:51.307707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:51.308198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:51.308638 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:51.309326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:51.309809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:51.310240 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:51.446286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:51.454640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bb6d2d8a-c4e1-4a83-a93f-44e7afcd49b4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.158167 Sep 27 04:23:51.455735 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 230/472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:51 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:52.478624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:52.479231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:52.485653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:52.486167 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:52.486604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:52.487292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:52.487743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:52.488170 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:52.617713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:52.625360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a57e6616-d212-428a-8265-bbb3755694c9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.151139 Sep 27 04:23:52.625951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 243/473] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:52 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:53.648243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:53.648851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:53.655185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:53.655677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:53.656154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:53.656846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:53.657305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:53.657805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:53.789598 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:53.797263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9419e889-5c9f-4511-9edc-629721315a0c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.152839 Sep 27 04:23:53.797846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 231/474] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:53 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:54.822260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:54.823048 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:54.829540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:54.830053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:54.830486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:54.831168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:54.831615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:54.832027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:54.959948 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:54.969590 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f1c3015c-2c46-4da5-a794-3002e5bf0aa8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.151921 Sep 27 04:23:54.970189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 244/475] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:54 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:55.993394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:55.994020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:56.000293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:56.000769 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:56.001197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:56.001905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:56.002362 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:56.002782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:56.127627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:56.136041 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7f4ad220-fdaa-4d2f-8681-f168e85acc1b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1675 microversion: 2.1 time: 0.146523 Sep 27 04:23:56.136611 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 232/476] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:55 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1675 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:57.159757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:57.160371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:57.167980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:57.168572 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:57.169504 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:57.170156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:57.170730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:57.172550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:57.298991 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:57.306619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b66b4500-cee9-43e1-aa6e-f8df8ac4085a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1652 microversion: 2.1 time: 0.150899 Sep 27 04:23:57.307184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 245/477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:57 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1652 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:57.328680 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:23:57.335713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:57.336195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:57.336625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:57.358056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Going to try to start instance {{(pid=120424) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 04:23:57.397775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-309346d8-20c0-475e-95e9-1d02f967f852 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action" status: 202 len: 0 microversion: 2.1 time: 0.072840 Sep 27 04:23:57.398417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 233/478] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:23:57 2022] POST /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/action => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:23:57.406615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:57.407278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:57.413515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:57.414032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:57.414466 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:57.415150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:57.415608 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:57.416023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:57.543489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:57.551020 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9aa497e2-9051-4e6f-9d60-672f8dc8df65 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.148231 Sep 27 04:23:57.551584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 246/479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:57 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:23:58.576160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:58.576851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:58.583042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:58.583533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:58.583961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:58.584646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:58.585095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:58.585519 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:58.711065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:58.718557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e0af74a9-3f0a-43cd-abe5-b3ba68612d32 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.147480 Sep 27 04:23:58.719114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 234/480] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:58 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:23:59.741969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:23:59.742569 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:23:59.749001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:59.749478 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:59.750062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:59.750778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:23:59.751229 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:23:59.751651 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:23:59.772116 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:23:59.887989 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:23:59.895433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e4a20447-2634-4279-9f62-cc4972fdae4a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.157923 Sep 27 04:23:59.895992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 247/481] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:23:59 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:00.918539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:00.919136 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:00.925470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:00.925983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:00.926424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:00.927633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:00.928087 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:00.928507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:01.071528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:01.079113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1b781ded-62b3-4acd-858e-b341ace12bb3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.164607 Sep 27 04:24:01.079665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 235/482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:00 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:02.103790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:02.104428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:02.110787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:02.111285 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:02.111712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:02.112394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:02.112846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:02.113272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:02.238664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:02.246705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a2f6ac42-9260-43c4-b9e6-fba8c88f216a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.147342 Sep 27 04:24:02.247465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 248/483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:02 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:03.269584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:03.270217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:03.277068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:03.277553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:03.278461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:03.278701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:03.279159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:03.279579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:03.409411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:03.417331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0db6551a-371d-4ad2-952b-185a4111f917 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1661 microversion: 2.1 time: 0.151495 Sep 27 04:24:03.417976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 236/484] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:03 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1661 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:04.439753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:04.440359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:04.447230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:04.447697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:04.448138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:04.448853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:04.449305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:04.449733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:04.583471 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:04.591050 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cb95cbf5-a4ff-499d-a59a-b0cd9f97c1c4 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1665 microversion: 2.1 time: 0.155139 Sep 27 04:24:04.591611 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 249/485] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:04 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1665 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:07.659395 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:07.659891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:07.667496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:07.667985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:07.668430 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:07.669124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:07.669577 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:07.670038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:07.754866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lazy-loading 'services' on Instance uuid dc919c44-b977-44e9-9c89-e92bcf1cebe6 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:07.826471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a1a97550-65fd-41a9-89b6-eb10b375ede9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6/os-volume_attachments/e0a8a204-89bd-49f7-9254-950534df115c" status: 202 len: 0 microversion: 2.1 time: 0.170970 Sep 27 04:24:07.827127 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 237/486] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:24:07 2022] DELETE /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6/os-volume_attachments/e0a8a204-89bd-49f7-9254-950534df115c => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:24:09.098670 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:09.099828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:09.102085 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:09.107213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:09.107693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:09.108131 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:09.108811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:09.109267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:09.109691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:09.134520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:24:09.210630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-535d7570-7fc2-4d16-b86c-89dff92bb894 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 204 len: 0 microversion: 2.1 time: 0.115540 Sep 27 04:24:09.211212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 250/487] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:24:09 2022] DELETE /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:24:09.222036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:09.222745 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:09.229873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:09.230363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:09.230803 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:09.231488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:09.231944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:09.232366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:09.361293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:09.368978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3b759a4c-a72d-4796-a731-c7f6cb60769d tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1514 microversion: 2.1 time: 0.152060 Sep 27 04:24:09.369534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 238/488] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:09 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1514 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:10.393062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:10.393776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:10.400821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:10.401302 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:10.401737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:10.402448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:10.402905 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:10.403333 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:10.534042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:10.542027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-02dcb080-101d-445b-ab1d-115cf8b39aeb tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1514 microversion: 2.1 time: 0.153149 Sep 27 04:24:10.542588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 251/489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:10 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:11.564558 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:11.565164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:11.571796 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:11.572263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:11.572699 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:11.573397 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:11.573873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:11.574294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:11.578757 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:11.874118 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '782ebdc88e21483eb3cecd00415b9b07', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '14d42cf78c32468cb8bf10aa847c7cc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:11.882469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-020df742-9980-4fec-bbe2-5f6e7a3fb2db tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 200 len: 1514 microversion: 2.1 time: 0.321650 Sep 27 04:24:11.884014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 239/490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:11 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 1514 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:12.111735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc919c44-b977-44e9-9c89-e92bcf1cebe6", "name": "network-vif-deleted", "tag": "0fe69958-0631-4361-8934-1da42bed69db"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:12.125193 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:12.125667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:12.126129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:12.139324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] Creating event network-vif-deleted:0fe69958-0631-4361-8934-1da42bed69db for instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 on hv-cinder-85918 Sep 27 04:24:12.145519 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-384f9aea-1f19-4043-88b1-4ce31134417d req-660200e1-4670-4113-8436-db3978a64b30 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036854 Sep 27 04:24:12.146169 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 252/491] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:12.907518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:12.908107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] [instance: dc919c44-b977-44e9-9c89-e92bcf1cebe6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:12.915174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:12.915656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:12.916095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:12.916778 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:12.917228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:12.917645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:12.930398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] HTTP exception thrown: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Request method failure captured: Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 HTTP/1.1 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnpkzET_rz6RAox7lvhcBQwYD-2NVrhJbKJHRpCoo4tXt4pbbdnTbbVArHX-P_LcErSAjK67BhREHz9CsgdZRZJuO35MLpyhdegYUtRU8V3-ZsKENmqU18blhGZeagCgOScYuP5SWpZtnWUJBvOGXWa2CjbppTIehLslu3x6EVjW4qc Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 14d42cf78c32468cb8bf10aa847c7cc3 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:24:12.931995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_14d42cf78c32468cb8bf10aa847c7cc3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/14d42cf78c32468cb8bf10aa847c7cc3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 14d42cf78c32468cb8bf10aa847c7cc3 Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-699738051 Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesSnapshotTestJSON-699738051-project Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 782ebdc88e21483eb3cecd00415b9b07 Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesSnapshotTestJSON-699738051-project Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. Sep 27 04:24:12.933534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'dc919c44-b977-44e9-9c89-e92bcf1cebe6'} Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:12.934400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. Sep 27 04:24:12.935877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:12.937113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] Returning 404 to user: Instance dc919c44-b977-44e9-9c89-e92bcf1cebe6 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:12.937113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bbf2e9ac-31f2-439e-8d7b-cbe6994c4bb9 tempest-VolumesSnapshotTestJSON-699738051 tempest-VolumesSnapshotTestJSON-699738051-project] 10.222.0.26 "GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6" status: 404 len: 111 microversion: 2.1 time: 0.029903 Sep 27 04:24:12.937113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 240/492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:12 2022] GET /compute/v2.1/servers/dc919c44-b977-44e9-9c89-e92bcf1cebe6 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:24:17.545973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:17.546678 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:17.553952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.554436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.554870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.555561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.556016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.556441 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.589501 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b72dac85-8d5a-48ff-9fee-b3ac56313915 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments/b6d46837-3522-41ab-8103-74de1fa63231" status: 200 len: 194 microversion: 2.20 time: 0.047845 Sep 27 04:24:17.590326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 253/493] 10.222.0.26 () {60 vars in 1454 bytes} [Tue Sep 27 04:24:17 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments/b6d46837-3522-41ab-8103-74de1fa63231 => generated 194 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:24:17.661198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:17.662290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:17.664187 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=34 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:17.669297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.669828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.670259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.670950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.671405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.671824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.755514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lazy-loading 'services' on Instance uuid 333a79d8-9886-4cc6-bbd4-335bee1d98b6 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:17.845675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5a2e16b7-4b46-4173-a58f-abd9c7277aac tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments/b6d46837-3522-41ab-8103-74de1fa63231" status: 202 len: 0 microversion: 2.20 time: 0.188502 Sep 27 04:24:17.846640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 241/494] 10.222.0.26 () {60 vars in 1457 bytes} [Tue Sep 27 04:24:17 2022] DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments/b6d46837-3522-41ab-8103-74de1fa63231 => generated 0 bytes in 190 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:24:17.855764 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:17.858406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:17.865437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.865965 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.866554 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.867260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:17.867746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:17.868146 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:17.899032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7526d152-91de-469b-ab56-e7c1a92bf7d1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047292 Sep 27 04:24:17.899874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 254/495] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 04:24:17 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 04:24:18.912292 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:18.914610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:18.921111 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:18.921597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:18.922073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:18.922759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:18.923215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:18.923638 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:18.952997 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-41781ba0-3213-458b-a32e-6b731210515b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.045421 Sep 27 04:24:18.956728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 242/496] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 04:24:18 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:19.016685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:19.017230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:19.024176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:19.024721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:19.025176 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:19.025899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:19.026438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:19.026773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:19.053106 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:24:19.126869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9180519e-783d-41ab-b8b5-d2f0023d9cca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 204 len: 0 microversion: 2.20 time: 0.113623 Sep 27 04:24:19.127654 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 255/497] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 04:24:19 2022] DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 04:24:19.139094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:19.139716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:19.145586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:19.146703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:19.147134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:19.147808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:19.148257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:19.148675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:19.284873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:19.293448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:19.294906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:19.296102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8752ef09-9abf-4a2b-92d7-e035e9f01560 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.162024 Sep 27 04:24:19.296650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 243/498] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:19 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:20.323495 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:20.324201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:20.331001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:20.331480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:20.332123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:20.332981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:20.333430 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:20.334046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:20.473669 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:20.481799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:20.483251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:20.484403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-566aa475-f3c6-482e-8c53-d810d2e0cb29 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.166938 Sep 27 04:24:20.484958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 256/499] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:20 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:21.511178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:21.511837 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:21.518596 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:21.519487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:21.520276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:21.521253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:21.521908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:21.522495 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:21.716197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:21.724222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:21.726862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:21.728112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e729d194-3daa-4475-b5c9-40d768b2b92e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1723 microversion: 2.20 time: 0.221122 Sep 27 04:24:21.728967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 244/500] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:21 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1723 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:22.025778 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "333a79d8-9886-4cc6-bbd4-335bee1d98b6", "name": "network-vif-deleted", "tag": "bbab3923-3dca-428e-93ca-c5dd6eb5cde9"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:22.036455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:22.036935 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:22.037370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:22.048773 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] Creating event network-vif-deleted:bbab3923-3dca-428e-93ca-c5dd6eb5cde9 for instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 on hv-cinder-85918 Sep 27 04:24:22.054230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-3bc520c4-1ed7-48b1-ac0f-42596cbc2b2e req-c5e94b24-0771-4316-81b5-51b77e68c3a4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032215 Sep 27 04:24:22.055014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 257/501] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:22.756251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:22.757449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:22.764145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:22.764754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:22.765416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:22.766274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:22.766827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:22.767506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:22.836612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:22.846611 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:22.848045 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:22.849253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a266696a-d8a7-4c68-92ba-b1caf64a8578 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1366 microversion: 2.20 time: 0.097131 Sep 27 04:24:22.850375 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 245/502] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:22 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1366 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:23.535422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cda190af-3ab1-4d13-98ac-fbcd5dc48d3d"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:23.538040 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:23.544258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:23.544743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:23.545184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:23.545902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:23.546393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:23.546822 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:23.874523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:23.875130 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:23.881462 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:23.881976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:23.882435 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:23.883110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:23.883558 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:23.884133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:23.947161 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:23.955307 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:23.956688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:23.957848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ff509bcf-b325-47e1-8faa-42e77087d1b8 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1366 microversion: 2.20 time: 0.087233 Sep 27 04:24:23.958400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 246/503] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:23 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1366 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:24:24.771491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3b2e12a0-3e73-45d9-b3fe-f6c922f86f2b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.240447 Sep 27 04:24:24.772088 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 258/504] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:24:23 2022] POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments => generated 194 bytes in 1243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:24.982994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:24.983711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:24.990581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:24.991067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:24.991506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:24.992189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:24.992709 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:24.993144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:25.023293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:25.023774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:25.024213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:25.024659 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lazy-loading 'fault' on Instance uuid 333a79d8-9886-4cc6-bbd4-335bee1d98b6 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:25.071408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:25.079405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:25.080826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:25.082015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9ddf9c59-9ead-4095-8ac7-bf110a50aa58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 200 len: 1371 microversion: 2.20 time: 0.102893 Sep 27 04:24:25.082566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 247/505] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:24 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 1371 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:26.107792 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:26.108505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:26.115215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:26.115703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:26.116134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:26.116812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:26.117266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:26.117684 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:26.130782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 HTTP/1.1 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:24:26.132390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:24:26.134094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '333a79d8-9886-4cc6-bbd4-335bee1d98b6'} Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:26.135024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:24:26.136500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:26.137851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:26.137851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3c2a8b0c-0553-469f-84b0-0f9564a56d77 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 404 len: 111 microversion: 2.20 time: 0.030805 Sep 27 04:24:26.137851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 259/506] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:26 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:24:26.142806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a69a93ac-6873-4381-a181-1dbfa49ccc7f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '.version_select at 0x7fce70b07c10>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:26.163201 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a69a93ac-6873-4381-a181-1dbfa49ccc7f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1173314002" status: 204 len: 0 microversion: 2.20 time: 0.024223 Sep 27 04:24:26.163765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 248/507] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 04:24:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1173314002 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 04:24:26.792900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bbb01144-7c75-4222-9180-0982cb237ad0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: .version_select at 0x7fce70902ca0>, body: {"keypair": {"name": "tempest-keypair-1623599026", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBArR7QeiIEIoJG2wtyJoc4AWMkwPD11+ZSwReeZif5DGTomN495Wnv2pKb7LJ6mXrKLtv2Sdzxks5x5wGZSI6rpX/DBC+w0UX+yl/sF8DsnwonFUYGW8jDVR5OYAHffObA=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:26.808934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-bbb01144-7c75-4222-9180-0982cb237ad0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:24:26.812645 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-bbb01144-7c75-4222-9180-0982cb237ad0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for user 7c56e7333ce4450b8cbd5caeafc73947 and project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:24:26.833656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bbb01144-7c75-4222-9180-0982cb237ad0 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.044715 Sep 27 04:24:26.834273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 260/508] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 04:24:26 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 46 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 04:24:28.369958 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-489517072", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "50cf116a-a1a9-4ea4-9151-a8bbdd56f0e6"}], "security_groups": [{"name": "tempest-securitygroup--94112026"}], "key_name": "tempest-keypair-1623599026", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:28.470015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] validate_networks() for [('50cf116a-a1a9-4ea4-9151-a8bbdd56f0e6', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:24:28.830806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:24:28.831183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:24:28.831662 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:24:28.832075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:24:28.832489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:24:28.841471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:28.841993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:28.842431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:28.843156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:28.843614 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:28.844053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:28.860540 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:24:28.864447 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Getting quotas for user 7c56e7333ce4450b8cbd5caeafc73947 and project 312fa89a4d9b43c5919b7bad5411002a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:24:28.875088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:24:28.877363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:24:28.924898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-60f8a03d-1b96-44f9-869e-cd22c6223b88 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 0.560436 Sep 27 04:24:28.925533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 249/509] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 04:24:28 2022] POST /compute/v2.1/servers => generated 400 bytes in 562 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 04:24:28.937138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:28.937750 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:28.993528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:29.000800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:29.002258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:29.003344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b7fee001-742e-45ba-a8c0-c64b6a4708e1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1281 microversion: 2.20 time: 0.070565 Sep 27 04:24:29.003897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 261/510] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:28 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1281 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:30.028927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:30.029589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:30.035844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:30.036331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:30.037382 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:30.038091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:30.038551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:30.038980 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:30.105600 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:30.117491 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:30.120879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:30.122291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-34049eeb-17db-4458-86c9-2d0ec378d116 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1333 microversion: 2.20 time: 0.097741 Sep 27 04:24:30.123578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 250/511] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:30 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1333 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:31.147234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:31.148009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:31.154650 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:31.155172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:31.155614 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:31.156341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:31.156806 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:31.157234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:31.160319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:31.295492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:31.304266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:31.305713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:31.306857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-51e21fd4-a6fc-4d9e-9079-c098c66d403e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.163524 Sep 27 04:24:31.307730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 262/512] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:31 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:31.671213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:31.690072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:31.690490 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:31.690927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:31.704174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] Creating event network-changed:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:24:31.710804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-1965828c-cb11-41c7-aaa8-d989f7d12423 req-57659d23-00f4-4918-9b0a-802dd77dc78e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043241 Sep 27 04:24:31.711423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 251/513] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:32.332010 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:32.332626 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:32.353873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.354402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.354869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.355583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.356046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.356479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.358470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:32.361780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:32.367465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.367962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.368408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.369102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.369561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.370015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.392732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'flavor' on Instance uuid 88ab8637-285a-4e7d-aac4-fe2fafab654e {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:32.486847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:32.495509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:32.496850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:32.497978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f8c46040-76bb-4bcb-9a2d-37121a75f2de tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.169889 Sep 27 04:24:32.498522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 263/514] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:32 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:32.551550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6adbebf2-c9ad-4eb7-8e70-da6074cfc1c8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/action" status: 200 len: 32 microversion: 2.1 time: 0.197715 Sep 27 04:24:32.552860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 252/515] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:24:32 2022] POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/action => generated 32 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:24:32.562676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:32.562676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:32.569557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.570144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.570527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.571216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:32.571668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:32.572090 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:32.697094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:32.705197 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-509ddfc9-2564-40a2-a5d2-f1595adb3b7b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.147013 Sep 27 04:24:32.705780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 264/516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:32 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:33.524179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:33.524869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:33.530968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:33.531457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:33.531891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:33.532579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:33.533035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:33.533456 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:33.729012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:33.729649 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:33.735999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:33.736480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:33.737108 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:33.737935 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:33.738570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:33.739173 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:33.750716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:33.763873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:33.767303 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:33.769312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-eb922f90-1640-42ce-9ec8-3826e9a20cc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.249916 Sep 27 04:24:33.770422 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 253/517] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:33 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:33.948906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:33.957536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ea40461c-6f4f-4062-890f-f1feecd9d93d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.232385 Sep 27 04:24:33.958832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 265/518] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:33 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:34.796196 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:34.796814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:34.803229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:34.803729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:34.804160 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:34.804857 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:34.805320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:34.805768 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:34.934837 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:34.943071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:34.945155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:34.946381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2cc56729-de30-49fa-ad9f-cd31e667f237 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.154179 Sep 27 04:24:34.947022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 254/519] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:34 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:34.980254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:34.980878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:34.987461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:34.987885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:34.988325 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:34.989016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:34.989475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:34.989953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:35.118536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:35.126651 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1035f616-0ea8-4238-8c2e-ad221ae21ce4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.150215 Sep 27 04:24:35.127257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 266/520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:34 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:35.974164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:35.975402 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:35.983062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:35.983731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:35.984751 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:35.985656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:35.986250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:35.986841 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.009926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1666472914"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:36.011112 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:36.017596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.018116 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.018556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.019248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.019708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.020305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.125527 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:36.129830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:36.136235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:36.137691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:36.139071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-54f4bcab-430d-4ddd-be34-915ed69fcf40 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.170003 Sep 27 04:24:36.139737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 255/521] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:35 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:36.158221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:36.158896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:36.164890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.network.security_group_api [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Removing security group e17d595b-0c19-4926-adc1-40062a9006bc from port 9d990fa5-a24c-4b14-8c63-0e0506b80887 Sep 27 04:24:36.166685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.167243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.167819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.168591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.169155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.169840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.310994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:36.319686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-639b1f69-7582-4c31-aa3a-3b6964e2de37 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.171923 Sep 27 04:24:36.320485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 256/522] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:36 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:36.503742 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7755ee57-363e-4512-8236-80bb314cd3fc tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.498272 Sep 27 04:24:36.505063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 267/523] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:24:36 2022] POST /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/action => generated 0 bytes in 500 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:24:36.514251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:36.529661 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.531308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.532323 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.546828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] Creating event network-changed:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:24:36.554144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-3bd465d9-9680-4373-abe2-c7dfc463da8a req-49fa22f0-46d5-4fd5-995d-07d320269127 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045849 Sep 27 04:24:36.555456 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 257/524] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:36.929261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "name": "network-vif-plugged", "status": "completed", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:36.946407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:36.947354 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:36.948872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:36.979023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] Creating event network-vif-plugged:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:24:36.988167 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-6816d1e2-3169-466e-915f-5dea107b9558 req-2080c4f7-aa8d-486f-8fcb-17cdae1f31bd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064559 Sep 27 04:24:36.989187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 268/525] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:37.172811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:37.175161 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:37.182990 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:37.183711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:37.184482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:37.186044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:37.186627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:37.187318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:37.343892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:37.345431 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:37.352518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:37.353004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:37.353435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:37.354221 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:37.354670 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:37.355305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:37.491537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:37.500513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8f3e764e-326a-47a2-bdaa-12cbf64c0b57 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.160842 Sep 27 04:24:37.501087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 269/526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:37 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:37.573469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:37.581713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:37.583176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:37.584786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-406ca85b-3cde-4a9b-b752-e3d0ef66abe5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.423241 Sep 27 04:24:37.597767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 258/527] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:37 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:38.345450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:38.350714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:38.355885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:38.367598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.368296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.368736 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.369436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.371189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.371834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.372658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.373089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.374306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.389339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] Creating event network-changed:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:24:38.406939 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-63b8fb77-2277-49b9-8a5e-4c31cdbc9fa3 req-88cffeeb-412c-41f3-9cd4-a3b73161f3be service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059619 Sep 27 04:24:38.408098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 259/528] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:38.493294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lazy-loading 'services' on Instance uuid c7036d25-7161-4e1f-95a5-5c8549139bc3 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:38.525092 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:38.525872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:38.534691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.535454 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.536269 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.536970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.537478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.538634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.598484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a08ca942-b245-445e-9cb3-5d8e2c226def tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments/923d96f0-0aee-4f37-bd78-b81672577dbb" status: 202 len: 0 microversion: 2.1 time: 0.253866 Sep 27 04:24:38.602881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 270/529] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:24:38 2022] DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments/923d96f0-0aee-4f37-bd78-b81672577dbb => generated 0 bytes in 262 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:24:38.608774 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:38.613882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:38.620910 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.621631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.622268 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.623137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.623738 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.624311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.662537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1d1ad8cc-87cb-411f-b8ca-b648e462f9e3 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058309 Sep 27 04:24:38.663492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 271/530] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:24:38 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:38.673404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:38.674321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:38.685358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.687317 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.688000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.690509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:38.691139 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:38.691719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:38.731010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:38.740719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7541a7b8-2294-4e90-8cfa-28ec5e5fcf65 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.220312 Sep 27 04:24:38.741533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 260/531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:38 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:38.919505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:38.929099 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:38.930622 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:38.931775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1937801d-1c1c-40e8-91d9-c7490ecfc236 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1405 microversion: 2.20 time: 0.262543 Sep 27 04:24:38.932576 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 272/532] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:38 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1405 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:24:39.677793 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:39.684484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:39.697904 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:39.700316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:39.702726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.703541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.704153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:39.705015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.706102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.707086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:39.770934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:39.771450 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-124a377b-9770-4eea-a077-1e1e6769d1ec tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.098854 Sep 27 04:24:39.771539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:39.772386 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 261/533] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:24:39 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:39.789339 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:39.791408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:39.793012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.793492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.795005 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:39.795894 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.799941 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.800616 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:39.956725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:39.957334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:39.965204 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.965698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.966159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:39.966851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:39.967304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:39.977050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.143347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:40.152250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-adaffa3d-78cf-4f58-83f6-ed7f19bfa887 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.390881 Sep 27 04:24:40.157990 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 273/534] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:39 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:40.177154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:40.202705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:40.210015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:40.212145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-cbfc95b9-17b2-4b8d-a4e9-435e1e60fd18 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1596 microversion: 2.20 time: 0.259315 Sep 27 04:24:40.222262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 262/535] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:24:39 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1596 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:24:40.238972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:40.239474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:40.245514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.246238 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.247000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.247854 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.248464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.249034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.368692 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0ad0308b-8510-46d6-bc73-66626eb2ce1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.134810 Sep 27 04:24:40.369284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 274/536] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 04:24:40 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-interface => generated 284 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:24:40.788057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:40.793208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:40.808355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.809026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.809615 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.811955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.812671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.814166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.852782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2d355d39-6341-438a-b2e7-3bbcef8da39d tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.072213 Sep 27 04:24:40.858992 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 263/537] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:24:40 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments => generated 25 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:24:40.867034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:40.868118 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:40.876376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.877215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.877831 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.878703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:40.879307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:40.881125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:40.982113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] HTTP exception thrown: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 is not attached to volume 923d96f0-0aee-4f37-bd78-b81672577dbb Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Request method failure captured: Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments/923d96f0-0aee-4f37-bd78-b81672577dbb HTTP/1.1 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnpBBVCLLFqwTW51vVF3aK-5VQodtS3ORSUsWk369vTroZl8EK1jynCCWfXTGRF1pAkGtJBQlATlM8-gNnh2dkVQ9LM3yN400j7g2Kyg2_20bCA_XxQJd9O0OXnbqSjmsSKZHhhHD5dlCL9089pYAltT3jd_WtuwgUcpTUJhSoSfuww Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 7e7b2ff0f0104769a550d0c0692ab713 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:24:40.985216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e7b2ff0f0104769a550d0c0692ab713", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 7e7b2ff0f0104769a550d0c0692ab713 Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-TestMinimumBasicScenario-1716668822-project Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 42d91d56296e4fdd917495df41401921 Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-TestMinimumBasicScenario-1716668822-project Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:24:40.986781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 is not attached to volume 923d96f0-0aee-4f37-bd78-b81672577dbb Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'server_id': 'c7036d25-7161-4e1f-95a5-5c8549139bc3', 'id': '923d96f0-0aee-4f37-bd78-b81672577dbb'} Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 923d96f0-0aee-4f37-bd78-b81672577dbb. Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:40.987592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 is not attached to volume 923d96f0-0aee-4f37-bd78-b81672577dbb Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 is not attached to volume 923d96f0-0aee-4f37-bd78-b81672577dbb Sep 27 04:24:40.988725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:40.991359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Returning 404 to user: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 is not attached to volume 923d96f0-0aee-4f37-bd78-b81672577dbb {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:40.994186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ad375016-68ea-40c7-be1b-de6086d40aea tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments/923d96f0-0aee-4f37-bd78-b81672577dbb" status: 404 len: 154 microversion: 2.1 time: 0.133432 Sep 27 04:24:40.999523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 275/538] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:24:40 2022] DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3/os-volume_attachments/923d96f0-0aee-4f37-bd78-b81672577dbb => generated 154 bytes in 139 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:24:41.184358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:41.185294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:41.191647 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:41.192431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:41.193185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:41.194126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:41.194741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:41.195324 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:41.619940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:41.632413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ed18764a-5e32-4001-82d9-8bab8032ceb2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.454695 Sep 27 04:24:41.633010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 264/539] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:41 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:41.919399 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:41.934429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:41.935188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:41.936295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:41.955512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] Creating event network-changed:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:24:41.963663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-01f0b3ee-338a-4b04-bbf1-382bcb110d15 req-b4d7eb16-f886-4b32-9480-9478700eb786 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047673 Sep 27 04:24:41.964914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 276/540] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:42.218262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:42.218879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:42.237418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.237953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.238395 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.239108 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.239565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.239993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.287086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:24:42.462839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ff838bec-c6c4-485f-a387-6c783b9e850a tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 204 len: 0 microversion: 2.1 time: 0.246890 Sep 27 04:24:42.468294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 265/541] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:24:42 2022] DELETE /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:24:42.478070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:42.479955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:42.493459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.494123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.494681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.496744 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.497308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.498035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.659746 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:42.661513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:42.670570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.671256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.671766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.672706 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:42.673279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:42.673795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:42.707174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:42.730882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8c6427c4-2503-4199-ad00-7a9703455d70 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1555 microversion: 2.1 time: 0.247645 Sep 27 04:24:42.731817 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 277/542] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:42 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1555 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:42.843739 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:42.859119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0534cdc5-82c8-4152-8ad3-7e81673bfe14 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.204018 Sep 27 04:24:42.861651 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 266/543] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:42 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:43.755169 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:43.756767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:43.764387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:43.765464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:43.766095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:43.767141 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:43.767966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:43.768558 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:43.891521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:43.892288 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:43.916799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:43.917616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:43.918312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:43.919180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:43.919858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:43.920956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:43.989314 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:43.998215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-71f559b1-7051-4451-963f-c95b41736e76 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1555 microversion: 2.1 time: 0.247313 Sep 27 04:24:44.008561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 278/544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:43 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1555 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:44.239336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:44.249209 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3e1efdde-0889-42d4-9315-453fb9552ee9 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.368731 Sep 27 04:24:44.250359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 267/545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:43 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:44.976925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7036d25-7161-4e1f-95a5-5c8549139bc3", "name": "network-vif-deleted", "tag": "9d990fa5-a24c-4b14-8c63-0e0506b80887"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:44.988563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:44.989077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:44.989480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.010431 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] Creating event network-vif-deleted:9d990fa5-a24c-4b14-8c63-0e0506b80887 for instance c7036d25-7161-4e1f-95a5-5c8549139bc3 on hv-cinder-85918 Sep 27 04:24:45.015615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-0618a4d9-c03a-4ac3-adea-8ef409580ab1 req-54423857-04ed-4d48-92c7-81686a40bcc8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042842 Sep 27 04:24:45.016851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 279/546] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:45.039275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:45.043133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:45.063266 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:45.099349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.031s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:45.100504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.101961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:45.102448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:45.102890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.161581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:45.167069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:45.167595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.168772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lazy-loading 'fault' on Instance uuid c7036d25-7161-4e1f-95a5-5c8549139bc3 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:45.270876 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42d91d56296e4fdd917495df41401921', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e7b2ff0f0104769a550d0c0692ab713', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:45.281056 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:45.284125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:45.295824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-40569848-4b99-44d2-b4eb-ad309cb29e74 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 200 len: 1349 microversion: 2.1 time: 0.267724 Sep 27 04:24:45.296679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 268/547] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:45 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 1349 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:45.316299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:45.316822 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:45.317278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.318093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:45.318752 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:45.324717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:45.535679 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:45.538846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:45.546459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ce6fc7aa-9ad3-4387-8b47-36c44475aed8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.273515 Sep 27 04:24:45.547247 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 280/548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:45 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:46.324867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.326658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] [instance: c7036d25-7161-4e1f-95a5-5c8549139bc3] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:46.339908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.345268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.345268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.345268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.345268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.346545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.363807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] HTTP exception thrown: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Request method failure captured: Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 HTTP/1.1 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnpBBVCLLFqwTW51vVF3aK-5VQodtS3ORSUsWk369vTroZl8EK1jynCCWfXTGRF1pAkGtJBQlATlM8-gNnh2dkVQ9LM3yN400j7g2Kyg2_20bCA_XxQJd9O0OXnbqSjmsSKZHhhHD5dlCL9089pYAltT3jd_WtuwgUcpTUJhSoSfuww Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 7e7b2ff0f0104769a550d0c0692ab713 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:24:46.378072 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e7b2ff0f0104769a550d0c0692ab713", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e7b2ff0f0104769a550d0c0692ab713"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 7e7b2ff0f0104769a550d0c0692ab713 Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1716668822 Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-TestMinimumBasicScenario-1716668822-project Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 42d91d56296e4fdd917495df41401921 Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-TestMinimumBasicScenario-1716668822-project Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. Sep 27 04:24:46.379622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'c7036d25-7161-4e1f-95a5-5c8549139bc3'} Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:46.380567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. Sep 27 04:24:46.382568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:46.384528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Returning 404 to user: Instance c7036d25-7161-4e1f-95a5-5c8549139bc3 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:46.385777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e807d4d5-a60a-4ec0-b2d0-43fe51f21569 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3" status: 404 len: 111 microversion: 2.1 time: 0.065425 Sep 27 04:24:46.386246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 269/549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:46 2022] GET /compute/v2.1/servers/c7036d25-7161-4e1f-95a5-5c8549139bc3 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:24:46.394899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9122e2c6-984e-4983-9b05-4b1899b1dc04 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] Calling method '.version_select at 0x7fce70a95b80>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.441934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9122e2c6-984e-4983-9b05-4b1899b1dc04 tempest-TestMinimumBasicScenario-1716668822 tempest-TestMinimumBasicScenario-1716668822-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1205956249" status: 202 len: 0 microversion: 2.1 time: 0.050879 Sep 27 04:24:46.447804 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 281/550] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 04:24:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1205956249 => generated 0 bytes in 58 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:24:46.499032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.501296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:46.508946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.509698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.510639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.511536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.512242 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.513876 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.571385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.571933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:46.577950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f9d7414b-08a4-4498-95d2-75062a51ae52 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.083049 Sep 27 04:24:46.581339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 270/551] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:24:46 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec/os-volume_attachments => generated 25 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:24:46.591746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.594130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.594661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.599421 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.602207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.602930 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.705516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.706146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:46.714925 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.715561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.716068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.717551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.718262 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.719457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.768784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:24:46.806850 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:46.843144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:46.852009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e1428542-f867-40e9-b53c-e772767015fd tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.285777 Sep 27 04:24:46.873303 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 282/552] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:46 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:46.896506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1a1ad47c-9d4d-4d43-97b7-4ac39636731f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 204 len: 0 microversion: 2.1 time: 0.194504 Sep 27 04:24:46.901106 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 271/553] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:24:46 2022] DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 04:24:46.906008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:46.906630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:46.914585 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.915475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.916360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:46.917821 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:46.918574 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:46.919181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:47.209844 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:47.226389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-505b397e-2675-41f6-a474-d2e4efa60c10 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1671 microversion: 2.1 time: 0.324137 Sep 27 04:24:47.227825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 283/554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:46 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1671 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:47.879870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:47.880585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:47.887435 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=90,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:24:47.890205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:47.890916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:47.891732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:47.892637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:47.893405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:47.894020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:48.253973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:48.254594 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:48.269324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:48.270205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:48.271172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:48.280875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:48.281463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:48.282063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:48.396382 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:48.429943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a550d74e-ff97-46d5-b937-e67a13de757a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.554299 Sep 27 04:24:48.446352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 272/555] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:47 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:48.550151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:48.558662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a3e14b0f-af74-4f1d-a5ae-a3f0927362ba tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1671 microversion: 2.1 time: 0.308524 Sep 27 04:24:48.573631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 284/556] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:48 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1671 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:49.380449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-429b4ce5-795b-4888-ba81-5ceb032f528a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Action: 'create', calling method: .version_select at 0x7fce7099af70>, body: {"keypair": {"name": "tempest-TestShelveInstance-1512923076", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLWUb7DycXMvxpLFIlDc//F6XyooXwjaz/oV6xX0u4s3SnITUQjMSZteI514LboQFzAwk2nV0j8aVM0kAiEHREkASni6ybjOpLZnW3XdOh/+A7IcVCbpTw4vDGUcKTXETQ=="}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:49.396536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-429b4ce5-795b-4888-ba81-5ceb032f528a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Getting quotas for project 1e595829629a4dd4a7fccbdabc896cb0. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:24:49.400973 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-429b4ce5-795b-4888-ba81-5ceb032f528a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Getting quotas for user 5ebc33b6c372434287b632d70f0a9e83 and project 1e595829629a4dd4a7fccbdabc896cb0. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:24:49.428465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-429b4ce5-795b-4888-ba81-5ceb032f528a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.328605 Sep 27 04:24:49.429060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 273/557] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:24:49 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:49.454307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:49.455002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:49.464870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:49.465358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:49.466187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:49.467058 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:49.467717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:49.468297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:49.585008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:49.587058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:49.594909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:49.595548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:49.596064 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:49.597290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:49.598939 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:49.601862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:49.802055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:49.838125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-14dd2b37-798a-4b96-b197-8ca4674e3e7d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.382765 Sep 27 04:24:49.855284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 285/558] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:49 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:49.995610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:50.044020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-46923526-db78-4538-9f45-4844088160e8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1671 microversion: 2.1 time: 0.458202 Sep 27 04:24:50.047631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 274/559] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:49 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1671 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:50.607146 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e24dba8d-ecd7-482f-9c93-79e714785fec", "name": "network-vif-deleted", "tag": "ef3b9eed-fccf-490a-bc0f-2fe41f79b621"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:50.631904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:50.632475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:50.632850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:50.650133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] Creating event network-vif-deleted:ef3b9eed-fccf-490a-bc0f-2fe41f79b621 for instance e24dba8d-ecd7-482f-9c93-79e714785fec on hv-cinder-85918 Sep 27 04:24:50.656897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-c4d3e98f-1ed9-4e77-b740-bd58751a487b req-779f2500-26ee-4422-ac22-1343ecacd6f1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054358 Sep 27 04:24:50.658424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 286/560] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:50.876593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:50.880173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:50.913573 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:50.918546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:50.919937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:50.920700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:50.921205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:50.921623 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:51.076349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:51.077278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:51.089397 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:51.090182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:51.094015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:51.095200 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:51.096053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:51.096769 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:51.211855 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:51.212351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:51.212805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:51.213277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'fault' on Instance uuid e24dba8d-ecd7-482f-9c93-79e714785fec {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:51.300659 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:51.321579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4b28d4b5-b803-4e88-84fa-2f31c0b0787d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1327 microversion: 2.1 time: 0.249979 Sep 27 04:24:51.322420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 287/561] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:51 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1327 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:51.420757 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:51.460394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8a8dbc87-2745-4ed7-b824-c134c6c2ba8b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1734 microversion: 2.1 time: 0.599305 Sep 27 04:24:51.461143 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 275/562] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:50 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1734 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:52.354271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:52.355172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:52.367153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.367919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.368906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.375219 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.376032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.376701 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.449900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.455245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.456743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.457225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'fault' on Instance uuid e24dba8d-ecd7-482f-9c93-79e714785fec {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:24:52.488512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:52.489636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:52.500552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.508802 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.509253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.513632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.514149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.514586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.573850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:52.583141 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cd37ccab-8972-4313-9934-a35f1287255b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 200 len: 1327 microversion: 2.1 time: 0.237650 Sep 27 04:24:52.586613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 288/563] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:52 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 1327 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:52.786313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:52.795869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4d6a256c-f5da-46a9-8614-1738b73e0ba8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1714 microversion: 2.1 time: 0.311343 Sep 27 04:24:52.796549 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 276/564] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:52 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1714 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:52.932274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:52.932873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:52.939069 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.939628 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.940133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:52.940929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:52.941456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:52.942030 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.058834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] HTTP exception thrown: Cannot 'detach_volume' instance 88ab8637-285a-4e7d-aac4-fe2fafab654e while it is in vm_state rescued Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Request method failure captured: Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments/cda190af-3ab1-4d13-98ac-fbcd5dc48d3d HTTP/1.1 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMno35C93UkxtOc6ZDX0FxmutNBu7fDaa-k2OG5GytQSAWRNNp4ten4pQn4RNoqEXm6vU4l2Pd-YXND1zBwqYnB2amEJy47FN9XvUHmMJF5xEqgAj58Sq_crQCMS5k-CWu6CeJD8FcZr32ZbDrVLzgLbfnG2CFqm0KMqdAFrML2D_IHw Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:24:53.060458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c25f11f92d4e4fd8965fe630ad77f3c3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 2d0ba2ae3ffe4bb9b99f1347719df411 Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:24:53.061846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Cannot 'detach_volume' instance 88ab8637-285a-4e7d-aac4-fe2fafab654e while it is in vm_state rescued Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'server_id': '88ab8637-285a-4e7d-aac4-fe2fafab654e', 'id': 'cda190af-3ab1-4d13-98ac-fbcd5dc48d3d'} Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:53.062506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 88ab8637-285a-4e7d-aac4-fe2fafab654e while it is in vm_state rescued Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 88ab8637-285a-4e7d-aac4-fe2fafab654e while it is in vm_state rescued Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.063646 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Returning 409 to user: Cannot 'detach_volume' instance 88ab8637-285a-4e7d-aac4-fe2fafab654e while it is in vm_state rescued {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:53.064586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-671cf0b0-be5e-4b49-a0cb-cc72aca8e56e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments/cda190af-3ab1-4d13-98ac-fbcd5dc48d3d" status: 409 len: 152 microversion: 2.1 time: 0.134207 Sep 27 04:24:53.064586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 289/565] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:24:52 2022] DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments/cda190af-3ab1-4d13-98ac-fbcd5dc48d3d => generated 152 bytes in 136 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 04:24:53.073155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:53.073773 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:53.081865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.082372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.082817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.083532 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.083996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.085161 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.166238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d8aa8b8f-c2a3-4862-9866-cddc5fb82010 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/action" status: 202 len: 0 microversion: 2.1 time: 0.098701 Sep 27 04:24:53.171067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 277/566] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:24:53 2022] POST /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/action => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 04:24:53.176099 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:53.176854 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:53.184115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.184877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.185911 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.186672 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.187986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.188587 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.480003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:53.491605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f809f411-6b30-4cad-bc18-b44ccb0853f7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1722 microversion: 2.1 time: 0.320173 Sep 27 04:24:53.492906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 290/567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:53 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1722 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:53.611911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:53.613294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:53.625615 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.626149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.626596 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.627516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:53.627988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:53.628423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:53.643621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec HTTP/1.1 Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:24:53.646183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:24:53.648007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'e24dba8d-ecd7-482f-9c93-79e714785fec'} Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:24:53.648930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:24:53.650444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:24:53.653090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:24:53.655261 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-914d38b0-7e69-40d9-b479-5be8f0604da8 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 404 len: 111 microversion: 2.1 time: 0.047845 Sep 27 04:24:53.667569 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9f2ddd9b-cad1-402c-8212-781852d3ae7f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '.version_select at 0x7fce7091cf70>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:53.672565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 278/568] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:53 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:24:53.703348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9f2ddd9b-cad1-402c-8212-781852d3ae7f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1427557109" status: 202 len: 0 microversion: 2.1 time: 0.040438 Sep 27 04:24:53.704408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 291/569] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 04:24:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1427557109 => generated 0 bytes in 42 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:24:54.518522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:54.519754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:54.532115 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:54.532640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:54.533253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:54.534006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:54.536933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:54.539418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:54.577047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-829621bc-2b37-427c-a01a-6540a6bc1728 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: .version_select at 0x7fce70902700>, body: {"keypair": {"name": "tempest-keypair-739774848", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC2dELBBB/1G+qAphPUubDvUmA5ddkDcDHQ2xdvhZVytTYy82nnyPIzzzizKQwfev8Sy5NtNUPzzlINPCvqddwRg4k1ZS+XoYGyco5/NezDL0XKnOWKSO20u5jqFbn0taw=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:54.632696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-829621bc-2b37-427c-a01a-6540a6bc1728 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:24:54.636096 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-829621bc-2b37-427c-a01a-6540a6bc1728 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for user f3438d850ce541cb9027ea3d5fe7dbf3 and project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:24:54.693424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-829621bc-2b37-427c-a01a-6540a6bc1728 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.120574 Sep 27 04:24:54.698183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 292/570] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:24:54 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:24:54.783170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:54.798770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-cb5fa0a7-e4a8-45b2-9533-21b54376b4eb tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1722 microversion: 2.1 time: 0.285598 Sep 27 04:24:54.799962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 279/571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:54 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1722 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:55.834487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:55.836341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:55.844147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:55.844893 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:55.845521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:55.847007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:55.847640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:55.848229 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:56.216561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:56.228872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-35f13e1e-c138-4e1d-bafd-fce77d2667e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1722 microversion: 2.1 time: 0.406213 Sep 27 04:24:56.229791 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 293/572] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:55 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1722 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:56.914620 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1686391214", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a72f6105-d258-45cb-a84d-18590bef93c2"}], "security_groups": [{"name": "tempest-securitygroup--2137855252"}], "key_name": "tempest-keypair-739774848", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:57.114937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] validate_networks() for [('a72f6105-d258-45cb-a84d-18590bef93c2', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:24:57.256185 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:57.257677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:57.267513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:57.269982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:57.272305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:57.273278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:57.274072 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:57.274719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:57.470976 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:57.481100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-71c8b8d1-985b-4925-88b8-bd1b78a87152 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1728 microversion: 2.1 time: 0.229241 Sep 27 04:24:57.482470 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 294/573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:57 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1728 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:24:57.515043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:57.516260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:57.523791 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:57.524620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:57.525305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:57.526239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:57.526856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:57.527447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:57.625542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4d59d25e-7d3f-4096-b2d7-da546c021033 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.114485 Sep 27 04:24:57.626257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 295/574] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:24:57 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-interface => generated 285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:57.977100 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:24:57.977855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:24:57.978662 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:24:57.993324 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:24:57.993324 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:24:58.000007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:58.000644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:58.001199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:58.002125 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:58.002910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:58.003591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:58.035328 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:24:58.040508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Getting quotas for user f3438d850ce541cb9027ea3d5fe7dbf3 and project 1c6cc70cbd2b4c04864f33ebafa06b5a. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:24:58.054689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:24:58.057511 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:24:58.110025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4529a59f-35a3-4dd1-ba8a-c6d325dd255f tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.199813 Sep 27 04:24:58.111552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 280/575] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:24:56 2022] POST /compute/v2.1/servers => generated 402 bytes in 1202 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:24:58.128695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-95a6e63c-54f9-4f27-8a45-a2ff5faa7dab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:58.129504 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-95a6e63c-54f9-4f27-8a45-a2ff5faa7dab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:58.304024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-95a6e63c-54f9-4f27-8a45-a2ff5faa7dab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:58.314426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-95a6e63c-54f9-4f27-8a45-a2ff5faa7dab tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1237 microversion: 2.1 time: 0.192675 Sep 27 04:24:58.315205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 296/576] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:58 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1237 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:24:59.222363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:24:59.232891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:59.233788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:59.234493 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:59.247012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] Creating event network-changed:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:24:59.253521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-b9c1010b-780a-4d8e-a177-41d45542b559 req-664384ef-68c2-4119-b745-ef529a638173 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035285 Sep 27 04:24:59.255167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 281/577] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:24:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:24:59.346389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:24:59.347041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:24:59.363059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:59.363972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:59.364490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:59.365294 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:24:59.366483 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:24:59.367081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:24:59.871307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:24:59.884123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b24d39b2-dbe4-4689-8b21-ddbe3bd3e0ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1289 microversion: 2.1 time: 0.542455 Sep 27 04:24:59.885057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 297/578] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:24:59 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1289 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:25:00.908251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:00.909036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:00.915782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:00.916535 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:00.917098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:00.917945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:00.918639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:00.919159 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:01.063227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:01.071616 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c859ee30-300b-4052-99ae-ebfcb057f1f3 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.168541 Sep 27 04:25:01.072179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 282/579] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:00 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:01.277661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:01.288991 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:01.289577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:01.290090 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:01.301232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] Creating event network-changed:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:25:01.307063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-b9c833bb-8c23-4038-afdd-418c69658b10 req-39625cbc-85a2-493d-8993-ad7ef4bb1238 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033088 Sep 27 04:25:01.307668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 298/580] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:02.095474 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:02.096136 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:02.102986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:02.103522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:02.103988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:02.104758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:02.105224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:02.105690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:02.299128 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:02.307232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7135045a-5e34-471b-b97e-f4112b76af23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.215880 Sep 27 04:25:02.307789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 283/581] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:02 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:03.323333 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "710d8936-179a-412b-9f1d-148e6648cebe", "tag": "4e861415-84ce-4923-aa54-770c8249a620"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:03.329350 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:03.329989 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:03.333419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:03.333975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:03.334958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:03.336921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:03.337399 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:03.337865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:03.338555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:03.339017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:03.339441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:03.345997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] Creating event network-changed:4e861415-84ce-4923-aa54-770c8249a620 for instance 710d8936-179a-412b-9f1d-148e6648cebe on hv-cinder-85918 Sep 27 04:25:03.351636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-8747b4c8-5f5b-4f79-aba6-5293d908f948 req-7c6bda9e-a100-4ddf-be49-12b59b6842fb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033039 Sep 27 04:25:03.352286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 299/582] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:03.713868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:03.721358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-047baf21-e508-4ccc-a485-b185b88b5408 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.395819 Sep 27 04:25:03.721938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 284/583] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:03 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:25:04.746018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:04.746715 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:04.752871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:04.753363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:04.753847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:04.754526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:04.754987 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:04.755412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:04.886439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:04.894977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-be1fe1b6-4ea9-454e-8cb0-cce33c815254 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.154236 Sep 27 04:25:04.895682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 300/584] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:04 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:06.021494 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:06.022656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:06.028553 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:06.029049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:06.029489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:06.030210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:06.030669 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:06.031095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:06.157846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:06.166244 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-65be1c0e-5b9c-4711-aa07-6f40c3af0918 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.251672 Sep 27 04:25:06.166804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 285/585] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:05 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:06.906804 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "710d8936-179a-412b-9f1d-148e6648cebe", "name": "network-vif-plugged", "status": "completed", "tag": "4e861415-84ce-4923-aa54-770c8249a620"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:06.922104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:06.923007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:06.923366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:06.940897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] Creating event network-vif-plugged:4e861415-84ce-4923-aa54-770c8249a620 for instance 710d8936-179a-412b-9f1d-148e6648cebe on hv-cinder-85918 Sep 27 04:25:06.947984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-06e02c2d-5ef8-4d29-b2f2-1b989532ceaa req-f839ddc5-52e4-40a5-90d3-f8331938a8fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045215 Sep 27 04:25:06.949112 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 301/586] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:07.190208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:07.190860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:07.197714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:07.198247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:07.198690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:07.199510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:07.200086 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:07.200628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:07.339451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:07.347387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8eba278f-8c37-4a0b-8b83-e0d60d061c33 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.162326 Sep 27 04:25:07.347952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 286/587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:07 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:07.548029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2041391819", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1512923076", "security_groups": [{"name": "tempest-secgroup-smoke-1351616088"}], "networks": [{"uuid": "41cd98f9-b206-4445-9870-d24633bca602"}], "block_device_mapping_v2": [{"uuid": "23524937-01f4-4263-b7fa-59abb32aa370", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:07.664758 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] validate_networks() for [('41cd98f9-b206-4445-9870-d24633bca602', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:25:08.064489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:25:08.065072 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:25:08.065370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:25:08.065800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:25:08.066213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:25:08.076984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:08.077393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:08.077934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:08.078728 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:08.079183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:08.079601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:08.095816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Getting quotas for project 1e595829629a4dd4a7fccbdabc896cb0. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:25:08.099692 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Getting quotas for user 5ebc33b6c372434287b632d70f0a9e83 and project 1e595829629a4dd4a7fccbdabc896cb0. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:25:08.158168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:25:08.160436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='23524937-01f4-4263-b7fa-59abb32aa370',volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:25:08.299215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-736ebe0e-ebb9-4287-9114-ec292e3a16d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.755299 Sep 27 04:25:08.299896 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 302/588] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:25:07 2022] POST /compute/v2.1/servers => generated 406 bytes in 757 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:25:08.311391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1f97ace6-a3ce-4583-a4bf-05e3b4a579c1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:08.312124 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1f97ace6-a3ce-4583-a4bf-05e3b4a579c1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:08.314249 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:08.315015 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=10 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:08.372728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1f97ace6-a3ce-4583-a4bf-05e3b4a579c1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:08.373401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:08.374445 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:08.380326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1f97ace6-a3ce-4583-a4bf-05e3b4a579c1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1089 microversion: 2.1 time: 0.072886 Sep 27 04:25:08.381290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 287/589] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:08 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1089 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:08.382277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:08.382851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:08.383514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:08.384716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:08.385666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:08.386174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:08.535478 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:08.544154 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-20de10e7-faad-4f40-9fd5-dc35814997df tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1363 microversion: 2.1 time: 0.175588 Sep 27 04:25:08.544736 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 303/590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:08 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:25:09.402787 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:09.403398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:09.412713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.413207 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.413646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.414879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.415337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.415796 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.479049 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:09.486572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a527e45a-305b-43c5-aab6-b8b037b4503e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1187 microversion: 2.1 time: 0.087589 Sep 27 04:25:09.487126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 288/591] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:09 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1187 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:09.568175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:09.568750 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:09.575082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.575565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.576002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.576688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.577146 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.577570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.592125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=100 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:09.715832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:09.726278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5702e395-b6a2-4064-a672-677aecee7570 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1548 microversion: 2.1 time: 0.161658 Sep 27 04:25:09.727540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 304/592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:09 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1548 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:09.749517 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:09.750103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:09.756247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.756941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.757381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.758140 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:09.758595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:09.759187 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:09.844333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-914b7532-388d-4f76-924c-1c0fd9b3974e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098645 Sep 27 04:25:09.845217 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 289/593] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:25:09 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:10.511385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:10.512038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:10.513924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:10.520126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:10.520621 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:10.521237 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:10.521978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:10.522437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:10.523142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:10.795824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:10.806432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bc35debc-6b50-468d-9eaf-102e82d262f4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.300274 Sep 27 04:25:10.807476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 305/594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:10 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:11.261443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "710d8936-179a-412b-9f1d-148e6648cebe", "tag": "4e861415-84ce-4923-aa54-770c8249a620"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:11.270986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:11.271484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:11.272440 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:11.283906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] Creating event network-changed:4e861415-84ce-4923-aa54-770c8249a620 for instance 710d8936-179a-412b-9f1d-148e6648cebe on hv-cinder-85918 Sep 27 04:25:11.289075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-b7337dfd-591d-4275-80b1-e466fc71b78e req-02f95cac-885e-4db1-ae9e-b0b7397b3150 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032314 Sep 27 04:25:11.289860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 290/595] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:11.457856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:11.479873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:11.481429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:11.483775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:11.498232 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] Creating event network-changed:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:25:11.507225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-07530e1e-1a3c-45bf-9564-6cc8b759d58b req-bd8347a8-0cfd-41a4-a1e2-79095006fc7e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055255 Sep 27 04:25:11.508424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 306/596] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:11.836156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:11.837235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:11.847591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:11.848096 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:11.848536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:11.849327 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:11.849950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:11.850374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:11.875044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:12.035061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:12.045910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f363255b-c291-4205-b31c-ed0887a93a96 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.217733 Sep 27 04:25:12.047670 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 291/597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:11 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:13.071267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:13.072035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:13.078715 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:13.079249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:13.079719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:13.080492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:13.081030 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:13.081465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:13.218396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:13.228001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ff531c62-1d23-4f85-97d9-472d789b2b98 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.161946 Sep 27 04:25:13.228663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 307/598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:13 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:14.253431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:14.254180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:14.262044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:14.262567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:14.263105 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:14.263881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:14.264410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:14.264901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:14.402354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:14.410458 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4832c4b1-c842-47cf-9a14-9d98a5cf13e4 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.161557 Sep 27 04:25:14.411484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 292/599] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:14 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:15.434382 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:15.434995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:15.441447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:15.441969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:15.442409 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:15.443094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:15.443545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:15.444087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:15.578731 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:15.586962 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bb4b40eb-83eb-4dd1-807b-b48544c711b7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.155805 Sep 27 04:25:15.587520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 308/600] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:15 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:16.608545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:16.609132 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:16.615389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:16.615875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:16.616316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:16.616993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:16.617986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:16.618412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:16.750176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:16.757645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-436ee800-aba8-45e2-a24c-6af15df36f4d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.153103 Sep 27 04:25:16.758770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 293/601] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:16 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:17.780472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:17.781089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:17.787301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:17.787789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:17.788227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:17.788917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:17.789899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:17.790328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:17.916520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:17.924043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fecc42b3-0190-41c4-9df2-76f4840d64d1 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.147676 Sep 27 04:25:17.924608 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 309/602] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:17 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:18.948141 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:18.948786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:18.951174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:18.951859 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=4 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:18.957950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:18.958444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:18.958885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:18.959580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:18.960039 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:18.960479 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:19.087432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:19.094908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1ad27ddd-bded-49ce-b941-df38eaa78a14 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.151433 Sep 27 04:25:19.095467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 294/603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:18 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:20.116863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:20.118077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:20.120013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:20.120660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=12 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:20.126779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:20.127263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:20.127696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:20.128371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:20.128820 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:20.129248 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:20.260565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:20.268391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-64ccd673-6006-4e89-8910-95d3ae6eb565 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.155306 Sep 27 04:25:20.269075 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 310/604] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:20 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:21.292272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:21.292993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:21.299465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:21.299952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:21.300394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:21.301079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:21.301539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:21.302000 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:21.441465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:21.449112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5e4f85bc-6c37-4b8e-b53d-bf2539960f33 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.161203 Sep 27 04:25:21.449675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 295/605] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:21 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:22.471742 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:22.472348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:22.478708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:22.479188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:22.479617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:22.480296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:22.480741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:22.481157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:22.608872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:22.616480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a43e0618-cd9a-485f-a098-9e57f1d70a4b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.148640 Sep 27 04:25:22.617044 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 311/606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:22 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:22.997419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "name": "network-vif-plugged", "status": "completed", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:23.007379 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:23.007881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:23.008317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:23.019279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] Creating event network-vif-plugged:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:25:23.024586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-f6fb3623-4e98-48e3-bfb0-da837642fa7b req-7d2d3cfe-aa16-4353-8efa-56991d9e8df2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030859 Sep 27 04:25:23.025672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 296/607] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:23.640140 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:23.640675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:23.647083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:23.647564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:23.648002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:23.649326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:23.649806 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:23.650236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:23.779402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:23.787196 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b5fb8724-a84a-4644-9354-e1cb63e98110 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.151034 Sep 27 04:25:23.787760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 312/608] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:23 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:24.810571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:24.811271 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:24.817845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:24.818335 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:24.818771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:24.819448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:24.819902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:24.820323 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:24.953143 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:24.960797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bc14989f-6889-4435-a024-a2f127f8054f tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1261 microversion: 2.1 time: 0.154455 Sep 27 04:25:24.961888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 297/609] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:24 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1261 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:25.983733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:25.984353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:25.990588 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:25.991259 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:25.991674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:25.992804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:25.993261 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:25.993686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:26.035870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:26.036688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:26.042707 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:26.043217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:26.043655 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:26.044345 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:26.044798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:26.045224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:26.068402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-96723216-e6e1-4147-9ca7-5da278ae7c19 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/ips" status: 200 len: 150 microversion: 2.20 time: 0.137576 Sep 27 04:25:26.069148 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 313/610] 10.222.0.26 () {60 vars in 1289 bytes} [Tue Sep 27 04:25:25 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/ips => generated 150 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:25:26.128366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:26.138835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9c3d033f-54b2-41de-aa02-f7b488e5a0e6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1446 microversion: 2.1 time: 0.158624 Sep 27 04:25:26.139886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 298/611] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:25 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1446 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:25:26.164958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:26.165563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:26.171488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:26.171974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:26.172404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:26.173092 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:26.173540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:26.173995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:26.313254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:26.323975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-30b5ac4e-778a-4444-8876-ad510984d2de tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1446 microversion: 2.1 time: 0.162877 Sep 27 04:25:26.325217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 314/612] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:26 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1446 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:27.864081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:27.864617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:27.871043 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:27.871537 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:27.871978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:27.872708 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:27.873126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:27.873555 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:27.895532 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lazy-loading 'flavor' on Instance uuid 5c6879a2-4ef5-4e16-8066-a2f05eca633c {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:28.087385 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a3de8a9f-1790-49d3-bab7-02572dc02b01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/action" status: 202 len: 0 microversion: 2.20 time: 0.227080 Sep 27 04:25:28.088170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 299/613] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 04:25:27 2022] POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/action => generated 0 bytes in 229 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:25:28.096202 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:28.096849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:28.104370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:28.105111 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:28.105559 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:28.106289 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:28.106955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:28.107383 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:28.239224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:28.250184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:28.250663 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:28.251121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:28.256730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:28.262295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] Creating event network-changed:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:25:28.266351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:28.267089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-d797906a-a64c-486c-b05e-4e6b95f35b32 req-ab56fcf4-8df8-45b3-a039-a02df16a9375 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031898 Sep 27 04:25:28.268103 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:28.268656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 300/614] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:28.269267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f1c8c661-7b69-4838-ad53-48d394d5c75f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.176919 Sep 27 04:25:28.277089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 315/615] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:28 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:29.296662 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:29.297587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:29.303541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:29.304023 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:29.304457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:29.305190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:29.305645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:29.306096 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:29.435098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:29.443142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:29.444574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:29.445713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6b6de4c4-a539-4acd-9da5-72fa6596fb58 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.154010 Sep 27 04:25:29.446285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 301/616] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:29 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:30.471270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:30.471843 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:30.479000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:30.479482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:30.479920 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:30.480603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:30.481061 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:30.481488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:30.610055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:30.618635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:30.620091 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:30.621246 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-09d9e515-cf32-437f-b5b6-fed32730d450 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.153956 Sep 27 04:25:30.621825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 316/617] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:30 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:31.646408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:31.647007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:31.653375 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:31.653947 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:31.654381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:31.655064 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:31.655513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:31.655937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:31.786980 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:31.795408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:31.796834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:31.798010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c2d3f0b6-66e8-47b2-8066-38fc2de68b04 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.155481 Sep 27 04:25:31.798550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 302/618] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:31 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:32.823217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:32.823819 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:32.830216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:32.830697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:32.831129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:32.831811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:32.832257 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:32.832680 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:32.960502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:32.968882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:32.970349 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:32.971494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-557bf988-1596-40d0-b7d0-c1cb17575132 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.152222 Sep 27 04:25:32.972051 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 317/619] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:32 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:33.997716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:33.998358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:34.004713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:34.005199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:34.005638 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:34.006349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:34.006807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:34.007236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:34.137510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:34.145446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:34.146935 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:34.148083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-aa9ce83c-cb86-4db9-a80f-b6f046dfbe00 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.154259 Sep 27 04:25:34.148636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 303/620] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:33 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:25:35.174914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:35.175515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:35.182463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:35.182940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:35.183373 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:35.184048 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:35.184493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:35.184910 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:35.310239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:35.318125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:35.319566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:35.320710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-575ce194-4772-4b66-aa62-3acd0d515dd1 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.149777 Sep 27 04:25:35.321262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 318/621] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:35 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:36.346728 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:36.347347 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:36.354256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:36.354741 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:36.355175 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:36.355858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:36.356316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:36.356738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:36.483744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:36.491610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:36.493643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:36.494824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-42c4d2dd-56d3-40b4-baa7-bc51ae39381a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.151986 Sep 27 04:25:36.495383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 304/622] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:36 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:37.090511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:37.091063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:37.097255 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.097736 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.098198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.098877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.099351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.099776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.179391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'services' on Instance uuid 88ab8637-285a-4e7d-aac4-fe2fafab654e {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:37.248377 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-16d80661-006e-4666-b60d-fa3a1effc08d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments/cda190af-3ab1-4d13-98ac-fbcd5dc48d3d" status: 202 len: 0 microversion: 2.1 time: 0.162191 Sep 27 04:25:37.249151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 319/623] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:25:37 2022] DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments/cda190af-3ab1-4d13-98ac-fbcd5dc48d3d => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 04:25:37.257406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:37.260169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:37.266184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.266652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.267146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.267902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.268314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.268737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.299436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d79dfadb-abd6-4764-a3b4-08cd26b1a15a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045811 Sep 27 04:25:37.299996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 305/624] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:25:37 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:37.520421 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:37.521122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:37.527338 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.527824 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.528265 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.528946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:37.529402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:37.529855 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:37.659853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:37.668126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:37.669563 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:37.670818 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-94aef5cf-6955-417b-ac05-7b234cfb0e28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.154646 Sep 27 04:25:37.671378 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 320/625] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:37 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:38.312024 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:38.314298 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:38.320714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.321239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.321683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.322400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.323027 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.323459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.353475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-84e25557-d285-4c2e-bafc-4cdb62ca36e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045403 Sep 27 04:25:38.354051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 306/626] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:25:38 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:25:38.426444 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:38.426956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:38.433357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.433883 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.434334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.435026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.435487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.435984 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.450535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:38.462289 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:25:38.534832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-655f48b0-fa58-494c-896a-7e8b0e9b96b4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 204 len: 0 microversion: 2.1 time: 0.112041 Sep 27 04:25:38.535404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 321/627] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:25:38 2022] DELETE /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:25:38.544748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:38.545442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:38.551331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.551811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.552244 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.552919 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.553370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.553842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.572162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:38.686602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:38.694389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c8558110-9947-442f-8b7f-5d4d878fc836 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1688 microversion: 2.1 time: 0.153450 Sep 27 04:25:38.694952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 307/628] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:38 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1688 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:38.697414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:38.698127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:38.703949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.704428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.704858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.705534 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:38.706007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:38.706427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:38.832348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:38.840922 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:38.842380 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:38.843525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0fcf6295-35d9-4ffb-b4e6-4c05923fe3ca tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.150834 Sep 27 04:25:38.844136 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 322/629] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:38 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:25:39.718179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:39.718789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:39.725243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:39.725733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:39.726200 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:39.726876 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:39.728003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:39.728431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:39.859846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:39.867710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-33272a71-1c9d-4118-86c1-c420d456413a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1688 microversion: 2.1 time: 0.153302 Sep 27 04:25:39.868280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 308/630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:39 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1688 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:39.868543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:39.869239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:39.875592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:39.876079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:39.876547 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:39.877240 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:39.877694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:39.878151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:40.003477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:40.011793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:40.013282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:40.014488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ebd5b4c0-1305-44d9-8faf-2d4dbb025f99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.149928 Sep 27 04:25:40.015050 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 323/631] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:39 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:40.890765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:40.891371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:40.897666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:40.898185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:40.898620 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:40.899337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:40.899791 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:40.900212 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:41.023992 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:41.031560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b91b6526-4d5c-4418-94ae-4c3adb5b2368 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1688 microversion: 2.1 time: 0.144629 Sep 27 04:25:41.032116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 309/632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:40 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1688 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:41.040760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:41.041370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:41.043973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:41.049052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:41.049544 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:41.050015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:41.050699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:41.051151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:41.051578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:41.183335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:41.191211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:41.192641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:41.193798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e18c43eb-a33f-44a4-84be-21f89c4dafd2 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.157730 Sep 27 04:25:41.194352 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 324/633] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:41 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:42.055916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:42.056547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:42.062927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:42.063416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:42.063855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:42.064543 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:42.065002 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:42.065432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:42.201461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:42.209943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-87ed65ea-561b-4620-9735-2b438dc34f1d tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1688 microversion: 2.1 time: 0.158473 Sep 27 04:25:42.210513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 310/634] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:42 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1688 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:42.217841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:42.218597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:42.224871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:42.225358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:42.225816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:42.226500 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:42.226954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:42.227377 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:42.429268 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:42.438159 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:42.439793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:42.441025 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ff51aead-e48f-4e07-a70b-ceebd1d85789 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.227156 Sep 27 04:25:42.441780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 325/635] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:42 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:43.183189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88ab8637-285a-4e7d-aac4-fe2fafab654e", "name": "network-vif-deleted", "tag": "8760ae9b-e8ce-45d6-a54c-90337fdcb044"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:43.195397 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.195928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.197019 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.209117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] Creating event network-vif-deleted:8760ae9b-e8ce-45d6-a54c-90337fdcb044 for instance 88ab8637-285a-4e7d-aac4-fe2fafab654e on hv-cinder-85918 Sep 27 04:25:43.214813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-13b66141-f1e8-4611-a8c5-bbd330b1410b req-1a086333-819f-4dcf-9a48-fc5e8a2bd589 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098719 Sep 27 04:25:43.215754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 311/636] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:43.232545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:43.241618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:43.252520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.253047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.253456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.254193 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.254974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.255387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.294052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.294536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.294977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.295418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'fault' on Instance uuid 88ab8637-285a-4e7d-aac4-fe2fafab654e {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:43.345201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:43.354602 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ab70a68a-d87c-44dd-9124-49093b57ade4 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 200 len: 1334 microversion: 2.1 time: 0.125729 Sep 27 04:25:43.355535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 326/637] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:43 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 1334 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:43.466797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:43.467649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:43.474186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.474680 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.475122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.475834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:43.476287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:43.476771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:43.610488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:43.619305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:43.621179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:43.622578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6438f25c-4bcd-4d46-942d-fb9ef2980864 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.160096 Sep 27 04:25:43.623309 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 312/638] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:43 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:44.378028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:44.378640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: 88ab8637-285a-4e7d-aac4-fe2fafab654e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:44.384844 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:44.385375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:44.385834 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:44.386513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:44.386964 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:44.387384 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:44.400728 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] HTTP exception thrown: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Request method failure captured: Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e HTTP/1.1 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMno35C93UkxtOc6ZDX0FxmutNBu7fDaa-k2OG5GytQSAWRNNp4ten4pQn4RNoqEXm6vU4l2Pd-YXND1zBwqYnB2amEJy47FN9XvUHmMJF5xEqgAj58Sq_crQCMS5k-CWu6CeJD8FcZr32ZbDrVLzgLbfnG2CFqm0KMqdAFrML2D_IHw Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:25:44.402368 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c25f11f92d4e4fd8965fe630ad77f3c3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 2d0ba2ae3ffe4bb9b99f1347719df411 Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. Sep 27 04:25:44.403942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '88ab8637-285a-4e7d-aac4-fe2fafab654e'} Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:25:44.404767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. Sep 27 04:25:44.406177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:25:44.407840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Returning 404 to user: Instance 88ab8637-285a-4e7d-aac4-fe2fafab654e could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:25:44.407840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6346bea1-d242-4741-9ae5-f2a1bdce6fb1 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e" status: 404 len: 111 microversion: 2.1 time: 0.029588 Sep 27 04:25:44.407840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 327/639] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:44 2022] GET /compute/v2.1/servers/88ab8637-285a-4e7d-aac4-fe2fafab654e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:25:44.412773 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-48eb85f5-72ea-4bc9-9e02-fcc2d18fea0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '.version_select at 0x7fce709851f0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:44.432317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-48eb85f5-72ea-4bc9-9e02-fcc2d18fea0b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-800992897" status: 202 len: 0 microversion: 2.1 time: 0.023402 Sep 27 04:25:44.432810 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 313/640] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:25:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-800992897 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:25:44.648471 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:44.650171 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:44.656260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:44.656947 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:44.657388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:44.658121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:44.658577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:44.659003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:44.786498 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:44.794665 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:44.796093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:44.797236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2107f412-17a7-47ad-b204-34d2720a5c01 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1741 microversion: 2.20 time: 0.152751 Sep 27 04:25:44.797790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 328/641] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:44 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1741 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:45.822489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:45.823095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:45.830017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:45.830502 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:45.830942 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:45.831625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:45.832076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:45.832496 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:45.969274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:45.977326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:45.979485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:45.981523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9bce0628-b15e-4da5-8241-057a95dc5ab3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1741 microversion: 2.20 time: 0.163320 Sep 27 04:25:45.982504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 314/642] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:45 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1741 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:47.007413 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:47.008026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:47.017953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.018704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.019333 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.020218 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.020850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.021441 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.173224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:47.185539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:47.187732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:47.189575 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1204b0fb-7da8-4e11-a936-e51cb68d3f28 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.185950 Sep 27 04:25:47.203259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 329/643] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:47 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:47.263964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "name": "network-vif-plugged", "status": "completed", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:47.282073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.284908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.288451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.306364 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:47.307094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] Creating event network-vif-plugged:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:25:47.309169 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:47.312516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-7d92d18d-ebd3-444d-8dd0-0227b80947d3 req-11c48f24-c0de-44f9-b8c1-65d2e7c85db5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.150650 Sep 27 04:25:47.316343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 315/644] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:47.317064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.317768 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.318240 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.318949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.319586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.320019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.350827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:25:47.459899 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d71685a5-96c5-401e-b33d-7ea6ce463df0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 204 len: 0 microversion: 2.1 time: 0.161430 Sep 27 04:25:47.461925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 330/645] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:25:47 2022] DELETE /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:25:47.471275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:47.472097 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:47.478586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.479389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.480117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.480914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:47.481686 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:47.482838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:47.633121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:47.641188 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d8260bf2-1efb-47b6-b187-1a37ef61be3e tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1508 microversion: 2.1 time: 0.174314 Sep 27 04:25:47.642041 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 316/646] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:47 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1508 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:48.221313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:48.222754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:48.231067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:48.231696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:48.232470 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:48.233355 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:48.234181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:48.235024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:48.377888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:48.386815 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:48.388584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:48.390334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-98b604df-54f6-423f-ba10-e278f110f319 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.173025 Sep 27 04:25:48.406104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 331/647] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:48 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:25:48.665277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:48.666240 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:48.673075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:48.673968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:48.674590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:48.675352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:48.675846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:48.676276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:48.814930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:48.823388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-071a8b06-b65f-4199-9397-033618eba2e2 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1508 microversion: 2.1 time: 0.162794 Sep 27 04:25:48.824046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 317/648] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:48 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1508 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:49.417069 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:49.417795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:49.424772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:49.425271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:49.425708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:49.426447 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:49.426904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:49.427331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:49.566535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:49.577137 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:49.579913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:49.581367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4f10338c-6098-4ccd-9808-4cc6f85684ea tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.168359 Sep 27 04:25:49.582319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 332/649] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:49 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:49.850283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:49.851506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:49.854621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=49,nova_api:DELETE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:49.861673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:49.862570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:49.863550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:49.864633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:49.865239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:49.865812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:50.022665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:50.031484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1691d051-c859-4b8d-b266-64d6544eb151 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1508 microversion: 2.1 time: 0.184921 Sep 27 04:25:50.032411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 318/650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:49 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1508 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:50.611451 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:50.612572 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:50.631143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:50.631644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:50.632077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:50.632795 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:50.633274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:50.633731 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:50.683615 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8c4e5f5-b02a-42c7-937e-d41da28f5fa9", "name": "network-vif-deleted", "tag": "b0f3948b-c804-4958-9c96-5fefbd31faea"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:50.698695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:50.699426 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:50.700507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:50.713194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] Creating event network-vif-deleted:b0f3948b-c804-4958-9c96-5fefbd31faea for instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 on hv-cinder-85918 Sep 27 04:25:50.718916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-ee2c33e0-5b7e-4a95-8014-accb9806b938 req-eedd0da9-a92b-4fd0-bd08-96e198e4f060 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039811 Sep 27 04:25:50.720142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 319/651] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:50.843530 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:50.864277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:50.865601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:50.866788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b3b57664-fc8b-4e36-bcf6-23c3208bf90f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.261655 Sep 27 04:25:50.867340 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 333/652] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:50 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:25:51.061984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:51.063278 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:51.069900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:51.070626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:51.071326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:51.072186 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:51.072795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:51.073362 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:51.112132 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:51.112839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:51.113506 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:51.114193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'fault' on Instance uuid d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:51.168840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:51.177975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-40a466f9-4b18-439d-84fe-519529e32f68 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 200 len: 1312 microversion: 2.1 time: 0.121994 Sep 27 04:25:51.178879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 320/653] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:51 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 1312 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:51.897356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:51.898611 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:51.905664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:51.906223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:51.906660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:51.907351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:51.907865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:51.908284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.055674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:52.065236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:52.066902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:52.068272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-659089f9-a6ad-429c-894d-2b218df9c878 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.175120 Sep 27 04:25:52.068833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 334/654] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:51 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:52.201430 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:52.202073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: d8c4e5f5-b02a-42c7-937e-d41da28f5fa9] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:52.209084 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.209500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.209978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.210706 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.211166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.211655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.226210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] HTTP exception thrown: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Request method failure captured: Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 HTTP/1.1 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMno35C93UkxtOc6ZDX0FxmutNBu7fDaa-k2OG5GytQSAWRNNp4ten4pQn4RNoqEXm6vU4l2Pd-YXND1zBwqYnB2amEJy47FN9XvUHmMJF5xEqgAj58Sq_crQCMS5k-CWu6CeJD8FcZr32ZbDrVLzgLbfnG2CFqm0KMqdAFrML2D_IHw Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:25:52.228564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c25f11f92d4e4fd8965fe630ad77f3c3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 2d0ba2ae3ffe4bb9b99f1347719df411 Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. Sep 27 04:25:52.229914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'd8c4e5f5-b02a-42c7-937e-d41da28f5fa9'} Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:25:52.230642 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. Sep 27 04:25:52.231795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:52.233481 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Returning 404 to user: Instance d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:25:52.234513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c6637365-b801-4636-a6ab-efff0187c723 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9" status: 404 len: 111 microversion: 2.1 time: 0.036975 Sep 27 04:25:52.235214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 321/655] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:52 2022] GET /compute/v2.1/servers/d8c4e5f5-b02a-42c7-937e-d41da28f5fa9 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:25:52.244890 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:52.245465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:52.254578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.255206 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.255704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.256682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.257236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.257736 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.261362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:52.299685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:25:52.387005 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4cd5fb62-1526-4089-a980-59e59184325a tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 204 len: 0 microversion: 2.1 time: 0.146518 Sep 27 04:25:52.387683 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 335/656] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:25:52 2022] DELETE /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:25:52.396378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:52.397079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:52.407762 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.409830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.410669 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.411566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:52.412146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:52.412931 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:52.563367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:52.584387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6929a206-a4a8-4b4b-aa37-3f80125157e8 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.192074 Sep 27 04:25:52.585512 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 322/657] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:52 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:25:53.096932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:53.097800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:53.105264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:53.105770 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:53.106214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:53.107086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:53.107711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:53.108286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:53.257227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:53.266676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:53.268385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:53.269549 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-029d52d5-68bc-4230-a876-b5f8cf7ce804 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.176873 Sep 27 04:25:53.285887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 336/658] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:53 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:53.609087 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:53.609637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:53.616400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:53.617097 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:53.617707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:53.619006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:53.619637 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:53.620218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:53.767836 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:53.778842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-456b3e89-7d27-42f3-bee9-c63242f990a0 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.173865 Sep 27 04:25:53.779446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 323/659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:53 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:54.297301 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:54.298579 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:54.306620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.307241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.308035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:54.308929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.309565 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.310199 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:54.452026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:54.460578 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:54.462772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:54.464211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3935099d-02fb-4ca1-8d89-a8121f13ea99 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.171088 Sep 27 04:25:54.482420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 337/660] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:54 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:54.802528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:54.803868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:54.811080 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.811743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.812355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:54.813210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.813856 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.814471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:54.960333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:54.969021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-39996430-0e41-4e7c-a13b-b5db43750559 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.170217 Sep 27 04:25:54.969621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 324/661] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:54 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:54.988330 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:54.988836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:54.996052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.996550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.996992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:54.997763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:54.998236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:54.998671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:55.027954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-063943e8-c325-4c38-8fc8-52b83597cd2c tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/ips" status: 200 len: 145 microversion: 2.1 time: 0.043852 Sep 27 04:25:55.028556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 338/662] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:25:54 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/ips => generated 145 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:55.489296 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:55.489966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:55.496909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:55.497697 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:55.498180 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:55.498928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:55.499583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:55.500022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:55.665012 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:55.675893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:55.677604 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:55.679079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9ce8d3a5-9111-48bd-bf44-e28df4acb70d tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1736 microversion: 2.20 time: 0.193746 Sep 27 04:25:55.694200 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 325/663] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:55 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1736 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:55.992263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:55.993493 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:55.999855 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.000348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.000788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.001474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.001963 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.002392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.127875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:56.135562 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1bedff53-38d4-4a2f-a296-1c68eefdfcab tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1520 microversion: 2.1 time: 0.147316 Sep 27 04:25:56.136122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 339/664] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:55 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1520 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:56.444572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b41bbf3e-fd9e-4f10-883b-904118a15981"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:56.447122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:56.453444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.453970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.454408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.455088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.455541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.455964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.700429 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:25:56.707768 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:56.708681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:56.719762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.720528 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.721334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.722954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:56.723752 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:56.724446 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:56.735982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d7cc3293-612c-4c15-9cdd-514f70d20faa tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.295045 Sep 27 04:25:56.737345 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 326/665] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:25:56 2022] POST /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 194 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:25:56.949451 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:56.958473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:56.960180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:56.961598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d96525f6-a06e-459e-ac06-f620a1927681 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1718 microversion: 2.20 time: 0.257986 Sep 27 04:25:56.962385 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 340/666] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:56 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1718 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:25:57.090371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e5e47978-60aa-4191-b5c0-f6421a4e7e97", "name": "network-vif-deleted", "tag": "b602518b-6faa-4bc5-b2ba-22e555f27363"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:57.102943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.103455 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.103887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:57.120805 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] Creating event network-vif-deleted:b602518b-6faa-4bc5-b2ba-22e555f27363 for instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 on hv-cinder-85918 Sep 27 04:25:57.128907 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-206255ed-2291-4508-83ce-e63156f4b8cb req-15751290-2918-4c53-a074-5bddaec9542a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044592 Sep 27 04:25:57.129631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 327/667] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:57.160951 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:57.161556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:57.169952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.170538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.171052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:57.171950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.172531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.173005 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:57.206401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.206880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.207327 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:57.207763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'fault' on Instance uuid e5e47978-60aa-4191-b5c0-f6421a4e7e97 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:57.250210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:57.257832 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-03613315-a8b9-47b0-8086-172e6e55d3d7 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1311 microversion: 2.1 time: 0.102909 Sep 27 04:25:57.259193 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 341/668] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:57 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1311 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:57.988443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:57.989141 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:57.996238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.996727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.997166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:57.997885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:57.998348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:57.998774 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:58.125246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:58.133387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:58.134929 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:58.136178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-df495d6c-3e78-41bc-9864-dd337074de89 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1718 microversion: 2.20 time: 0.151899 Sep 27 04:25:58.136734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 328/669] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:57 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1718 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:58.280443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:58.280967 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:58.287612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:58.288096 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:58.288535 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:58.289297 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:58.290804 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:58.291489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:58.324117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:58.324593 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:58.325029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:58.325465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lazy-loading 'fault' on Instance uuid e5e47978-60aa-4191-b5c0-f6421a4e7e97 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:25:58.367574 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2d0ba2ae3ffe4bb9b99f1347719df411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c25f11f92d4e4fd8965fe630ad77f3c3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:58.375711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-041efb6e-6add-49ec-b3ed-ceafa937e143 tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 200 len: 1311 microversion: 2.1 time: 0.099261 Sep 27 04:25:58.376265 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 342/670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:58 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 1311 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:25:59.160784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:59.161393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:59.168146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.168634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.169071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.169801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.170260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.170682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.182888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "name": "network-vif-unplugged", "status": "completed", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:59.194732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.195196 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.195624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.207443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] Creating event network-vif-unplugged:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:25:59.213666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-3a3256c3-1e0b-47ca-8825-f9b34c60cf60 req-625a5421-c588-408c-b368-fbc0b68d442f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036835 Sep 27 04:25:59.214814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 343/671] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:25:59.325295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:59.333941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:59.335403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:25:59.336700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-99fcd0a3-b756-4904-8a86-719122be4d1c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1721 microversion: 2.20 time: 0.180002 Sep 27 04:25:59.337432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 329/672] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:25:59 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1721 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:25:59.365913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c979818d-09ce-494e-93ba-0ed690017082"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:25:59.369101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:59.376875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.377360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.377818 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.378508 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.378955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.379374 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.398626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:59.399274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] [instance: e5e47978-60aa-4191-b5c0-f6421a4e7e97] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:59.405565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.408277 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.408732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.409446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.409960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.410393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.427580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] HTTP exception thrown: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Request method failure captured: Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 HTTP/1.1 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMno35C93UkxtOc6ZDX0FxmutNBu7fDaa-k2OG5GytQSAWRNNp4ten4pQn4RNoqEXm6vU4l2Pd-YXND1zBwqYnB2amEJy47FN9XvUHmMJF5xEqgAj58Sq_crQCMS5k-CWu6CeJD8FcZr32ZbDrVLzgLbfnG2CFqm0KMqdAFrML2D_IHw Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:25:59.429215 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c25f11f92d4e4fd8965fe630ad77f3c3", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c25f11f92d4e4fd8965fe630ad77f3c3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: c25f11f92d4e4fd8965fe630ad77f3c3 Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1742348020 Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 2d0ba2ae3ffe4bb9b99f1347719df411 Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1742348020-project Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. Sep 27 04:25:59.431329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'e5e47978-60aa-4191-b5c0-f6421a4e7e97'} Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:25:59.432158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. Sep 27 04:25:59.433423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:25:59.434592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] Returning 404 to user: Instance e5e47978-60aa-4191-b5c0-f6421a4e7e97 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:25:59.434592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-710f0e29-d543-434e-b2f9-c5029118042b tempest-ServerRescueNegativeTestJSON-1742348020 tempest-ServerRescueNegativeTestJSON-1742348020-project] 10.222.0.26 "GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97" status: 404 len: 111 microversion: 2.1 time: 0.035976 Sep 27 04:25:59.434592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 330/673] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:25:59 2022] GET /compute/v2.1/servers/e5e47978-60aa-4191-b5c0-f6421a4e7e97 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:25:59.670516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-298248ca-aa10-400f-a094-4869b450ecc4 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.309872 Sep 27 04:25:59.676660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 344/674] 10.222.0.26 () {62 vars in 1364 bytes} [Tue Sep 27 04:25:59 2022] POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments => generated 188 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 04:25:59.818567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:25:59.819216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:25:59.825690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.826301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.826816 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:25:59.827580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:25:59.828112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:25:59.828608 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:00.070158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-41b17180-24b6-4524-b7b8-11aef486ad5c tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments/c979818d-09ce-494e-93ba-0ed690017082" status: 202 len: 0 microversion: 2.20 time: 0.255389 Sep 27 04:26:00.070743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 331/675] 10.222.0.26 () {60 vars in 1457 bytes} [Tue Sep 27 04:25:59 2022] DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments/c979818d-09ce-494e-93ba-0ed690017082 => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 04:26:00.134181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:00.134787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:00.140714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:00.141206 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:00.141652 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:00.142398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:00.142858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:00.143293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:00.189888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120423) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 04:26:00.259938 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-14ac04ca-b4d1-4428-a4ec-ddbcbd156566 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/action" status: 202 len: 0 microversion: 2.20 time: 0.130250 Sep 27 04:26:00.260743 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 345/676] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 04:26:00 2022] POST /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 04:26:00.268952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:00.269982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:00.280290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:00.280892 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:00.281408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:00.282442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:00.283020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:00.283528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:00.667595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:00.676868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:00.678403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:00.679630 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-89eab545-728d-48e1-9d84-20a509671987 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1669 microversion: 2.20 time: 0.414799 Sep 27 04:26:00.680451 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 332/677] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:00 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1669 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 04:26:01.706053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:01.707305 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:01.718037 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:01.721121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:01.723215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:01.724341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:01.726453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:01.727510 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:01.959685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:01.965656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=6,nova_cell1:INSERT=5 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:01.977087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:01.979154 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:01.980328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bf18f20a-6db7-4fec-bdfc-1080b2c80e4f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.278358 Sep 27 04:26:01.981518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 346/678] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:01 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:03.009619 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:03.011021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:03.018797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:03.019676 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:03.020279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:03.021156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:03.021785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:03.022362 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:03.347673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:03.358713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:03.361112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:03.363210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d8a35ceb-1147-482b-a2c3-c11eac425749 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.360242 Sep 27 04:26:03.380552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 333/679] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:03 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:04.393225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:04.393873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:04.403132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:04.420740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:04.420740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:04.420740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:04.420740 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:04.433570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:04.597784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:04.607177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:04.608664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:04.609859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-18601528-fcbd-479d-ae2f-62963fb3a956 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.221076 Sep 27 04:26:04.610415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 347/680] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:04 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:04.736200 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-572cd82f-aa54-404f-b586-4005dc2a3dd6 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Action: 'create', calling method: .version_select at 0x7fce70b07f70>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-311711590"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:05.053643 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-572cd82f-aa54-404f-b586-4005dc2a3dd6 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.457068 Sep 27 04:26:05.054571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 334/681] 10.222.0.26 () {60 vars in 1155 bytes} [Tue Sep 27 04:26:04 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 459 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 27 04:26:05.067229 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cbfca311-8324-4329-971b-5b326b168230 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d6dc0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:05.123441 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cbfca311-8324-4329-971b-5b326b168230 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 200 len: 344 microversion: 2.1 time: 0.059847 Sep 27 04:26:05.123753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 348/682] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:05 2022] GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 344 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:26:05.637248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:05.637855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:05.644564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:05.645052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:05.645482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:05.646191 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:05.646646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:05.647073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:05.778455 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:05.786978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:05.789237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:05.791294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-827aec79-b02d-43ad-8b1c-9bc8dc2f5a5f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.158618 Sep 27 04:26:05.791887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 335/683] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:05 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:06.137198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ead4630c-34c7-4e89-b93b-736027277ca8 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce70a958b0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:06.198526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ead4630c-34c7-4e89-b93b-736027277ca8 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 200 len: 345 microversion: 2.1 time: 0.065546 Sep 27 04:26:06.207816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 349/684] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:06 2022] GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 345 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:06.213267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c4f9c4d9-a74c-4846-bea4-cad9b0aa9230 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Action: 'create', calling method: .version_select at 0x7fce70b07b80>, body: {"snapshot": {"volume_id": "246fdf5f-6978-4e44-ab82-fe71297a8b18", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1512210719"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:06.382667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c4f9c4d9-a74c-4846-bea4-cad9b0aa9230 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.173112 Sep 27 04:26:06.385343 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 336/685] 10.222.0.26 () {60 vars in 1162 bytes} [Tue Sep 27 04:26:06 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:26:06.399162 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5011c3fb-cc09-47dd-92a0-269329c870f7 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d7160>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:06.437707 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5011c3fb-cc09-47dd-92a0-269329c870f7 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.044029 Sep 27 04:26:06.438622 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 350/686] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:06 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:06.450299 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7f76877e-77ec-4613-a206-69c8ec3039ad tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce70985160>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:06.503231 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7f76877e-77ec-4613-a206-69c8ec3039ad tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.057487 Sep 27 04:26:06.504071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 337/687] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 04:26:06 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:06.516846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-475f90c5-570a-4eee-b87b-d6f268ef85d4 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce70a95550>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:06.548706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-475f90c5-570a-4eee-b87b-d6f268ef85d4 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.036074 Sep 27 04:26:06.549371 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 351/688] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:06 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:26:06.818957 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:06.819672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:06.827407 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:06.828499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:06.829224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:06.830336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:06.831183 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:06.831965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:06.964468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:06.972541 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:06.974033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:06.975222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d79a4c60-cab9-4cbe-a349-a473afc5286a tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.162321 Sep 27 04:26:06.975780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 338/689] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:06 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:07.566003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-eb4b7ed5-0a6b-41da-810f-8b860565f382 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d69d0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:07.597317 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-eb4b7ed5-0a6b-41da-810f-8b860565f382 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.038130 Sep 27 04:26:07.598693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 352/690] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:07 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:08.000526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:08.001133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:08.007689 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:08.008169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:08.009149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:08.009861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:08.010314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:08.010737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:08.164875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:08.174882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:08.176840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:08.179060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f3d66a78-127a-4375-b492-97de03a2512f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.182425 Sep 27 04:26:08.180963 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 339/691] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:07 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:08.611016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-21911afd-a11d-442f-8e01-f4a73e6ac4de tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d6ee0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:08.651133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-21911afd-a11d-442f-8e01-f4a73e6ac4de tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.043847 Sep 27 04:26:08.651771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 353/692] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:08 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:09.205683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:09.207181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:09.215777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:09.216897 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:09.217646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:09.218900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:09.219654 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:09.220909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:09.363342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:09.371270 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:09.372826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:09.374066 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1c2c9d89-1f06-432f-ae62-ac358e308019 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.172232 Sep 27 04:26:09.374626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 340/693] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:09 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:09.664735 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b01bf452-a66a-4dd0-b8db-ef11c630dc4c tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce7093daf0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:09.701436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b01bf452-a66a-4dd0-b8db-ef11c630dc4c tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.041193 Sep 27 04:26:09.707502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 354/694] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:09 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:10.400457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:10.401059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:10.408151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:10.408650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:10.409085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:10.409997 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:10.410241 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:10.410848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:10.559509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:10.568938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:10.570720 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:10.571938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b216da6d-a1e1-4d51-800f-88cb1500c986 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.175399 Sep 27 04:26:10.572497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 341/695] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:10 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:10.715377 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3db09980-85bf-45b9-a891-3a2e213c1519 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d7ca0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:10.766961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3db09980-85bf-45b9-a891-3a2e213c1519 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 288 microversion: 2.1 time: 0.055920 Sep 27 04:26:10.767729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 355/696] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:10 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 288 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:10.779297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9529d8d4-67c3-41ed-8543-4135342a6ff9 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce70a4e4c0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:10.836803 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9529d8d4-67c3-41ed-8543-4135342a6ff9 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "DELETE /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 202 len: 0 microversion: 2.1 time: 0.061674 Sep 27 04:26:10.838123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 342/697] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 04:26:10 2022] DELETE /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:26:10.847136 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-76a8bcf5-daa0-475d-8219-209af279a11f tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d03a0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:10.879480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-76a8bcf5-daa0-475d-8219-209af279a11f tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 200 len: 287 microversion: 2.1 time: 0.036101 Sep 27 04:26:10.885397 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 356/698] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:10 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:11.600263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:11.600826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:11.607022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:11.607514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:11.607966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:11.608681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:11.609142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:11.609572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:11.740303 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:11.748384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:11.749857 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:11.751018 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-43dbb4ac-2603-413d-9e70-9db00f667a57 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.155217 Sep 27 04:26:11.751580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 343/699] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:11 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:11.892766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e2225213-05dc-4883-9808-50524f14ec27 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce7093d040>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:11.917993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-e2225213-05dc-4883-9808-50524f14ec27 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] HTTP exception thrown: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e2225213-05dc-4883-9808-50524f14ec27 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Request method failure captured: Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 HTTP/1.1 Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntcoTRo2sFQJe6aNGoSEXxcqt9vV9vPcE4YKTrdGEPYTIm7oWoG0wq0ZNXlqf-fGoNe1S-FatoKRoE6n550jbInzTycZF5-i6xt0oNaShojzCZAE6yuFA4ll4dd1QkPzRR1egwiJE0EEuxwtL-uvueNcCbFrkrazkCnPuAasdFx59s Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:11.921343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2356c8895ab41f68983fc8328fad8cc", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: bcbf6702a9064ca48776744cce268284 Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'd4c5d33e-a7fd-40c4-933e-b66889a738a0'} Sep 27 04:26:11.923038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. (HTTP 404) (Request-ID: req-eca501d4-bcf7-45be-b84f-a1405017ad96) Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.924150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:11.925598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 27 04:26:11.927004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e2225213-05dc-4883-9808-50524f14ec27 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Returning 404 to user: Snapshot d4c5d33e-a7fd-40c4-933e-b66889a738a0 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:11.928460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e2225213-05dc-4883-9808-50524f14ec27 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0" status: 404 len: 111 microversion: 2.1 time: 0.034346 Sep 27 04:26:11.929488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 357/700] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 04:26:11 2022] GET /compute/v2.1/os-snapshots/d4c5d33e-a7fd-40c4-933e-b66889a738a0 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:11.931967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7e4f2567-a56c-4883-b7da-27baf09ee09a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce70905670>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:11.994760 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7e4f2567-a56c-4883-b7da-27baf09ee09a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 202 len: 0 microversion: 2.1 time: 0.066160 Sep 27 04:26:11.995426 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 344/701] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:26:11 2022] DELETE /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:26:12.003750 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b5d45b47-e26a-4fc5-a00c-9ec66002a22e tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707d0c10>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:12.055234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b5d45b47-e26a-4fc5-a00c-9ec66002a22e tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 200 len: 344 microversion: 2.1 time: 0.055145 Sep 27 04:26:12.055881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 358/702] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:11 2022] GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 344 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:12.777433 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:12.843312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:12.843312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:12.843312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:12.843312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:12.843780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:12.843780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:12.843780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:12.925729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:12.933872 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:12.935194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:12.936342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e4b5dde2-584b-421b-92ba-cfa9c70894dd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.163855 Sep 27 04:26:12.936898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 345/703] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:12 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:13.071388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-08d2de00-f28f-4807-9f27-6960af98f24b tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce708cc940>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:13.098861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-08d2de00-f28f-4807-9f27-6960af98f24b tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] HTTP exception thrown: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-08d2de00-f28f-4807-9f27-6960af98f24b tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Request method failure captured: Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 HTTP/1.1 Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntcoTRo2sFQJe6aNGoSEXxcqt9vV9vPcE4YKTrdGEPYTIm7oWoG0wq0ZNXlqf-fGoNe1S-FatoKRoE6n550jbInzTycZF5-i6xt0oNaShojzCZAE6yuFA4ll4dd1QkPzRR1egwiJE0EEuxwtL-uvueNcCbFrkrazkCnPuAasdFx59s Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:13.101125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2356c8895ab41f68983fc8328fad8cc", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: bcbf6702a9064ca48776744cce268284 Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '246fdf5f-6978-4e44-ab82-fe71297a8b18'} Sep 27 04:26:13.102682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. (HTTP 404) (Request-ID: req-c24655a6-33bc-4f28-85f0-7e561c34b3b0) Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.104046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:13.105183 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 04:26:13.106318 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-08d2de00-f28f-4807-9f27-6960af98f24b tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Returning 404 to user: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-08d2de00-f28f-4807-9f27-6960af98f24b tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 404 len: 109 microversion: 2.1 time: 0.035924 Sep 27 04:26:13.107416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 359/704] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:13 2022] GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:13.113402 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-289a2a36-a61c-4f69-abbb-1222847ce059 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce708cc1f0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:13.141726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-289a2a36-a61c-4f69-abbb-1222847ce059 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] HTTP exception thrown: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-289a2a36-a61c-4f69-abbb-1222847ce059 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Request method failure captured: Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: DELETE /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 HTTP/1.1 Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMntcoTRo2sFQJe6aNGoSEXxcqt9vV9vPcE4YKTrdGEPYTIm7oWoG0wq0ZNXlqf-fGoNe1S-FatoKRoE6n550jbInzTycZF5-i6xt0oNaShojzCZAE6yuFA4ll4dd1QkPzRR1egwiJE0EEuxwtL-uvueNcCbFrkrazkCnPuAasdFx59s Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:13.144965 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2356c8895ab41f68983fc8328fad8cc", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: bcbf6702a9064ca48776744cce268284 Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '246fdf5f-6978-4e44-ab82-fe71297a8b18'} Sep 27 04:26:13.146509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. (HTTP 404) (Request-ID: req-a697fd9f-60a2-4d72-a10d-a454f206f1eb) Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.147366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:13.148545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 04:26:13.149673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-289a2a36-a61c-4f69-abbb-1222847ce059 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Returning 404 to user: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-289a2a36-a61c-4f69-abbb-1222847ce059 tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 404 len: 109 microversion: 2.1 time: 0.036872 Sep 27 04:26:13.151423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 346/705] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:26:13 2022] DELETE /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:13.154668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-be60d639-6806-4e3d-b4f0-3251603a9b0a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Calling method '.version_select at 0x7fce707a74c0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:13.185342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-be60d639-6806-4e3d-b4f0-3251603a9b0a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] HTTP exception thrown: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-be60d639-6806-4e3d-b4f0-3251603a9b0a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Request method failure captured: Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 HTTP/1.1 Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntcoTRo2sFQJe6aNGoSEXxcqt9vV9vPcE4YKTrdGEPYTIm7oWoG0wq0ZNXlqf-fGoNe1S-FatoKRoE6n550jbInzTycZF5-i6xt0oNaShojzCZAE6yuFA4ll4dd1QkPzRR1egwiJE0EEuxwtL-uvueNcCbFrkrazkCnPuAasdFx59s Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:13.187278 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2356c8895ab41f68983fc8328fad8cc", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2356c8895ab41f68983fc8328fad8cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: e2356c8895ab41f68983fc8328fad8cc Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1741608810 Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: bcbf6702a9064ca48776744cce268284 Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1741608810-project Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '246fdf5f-6978-4e44-ab82-fe71297a8b18'} Sep 27 04:26:13.188636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. (HTTP 404) (Request-ID: req-b1e98f48-21eb-482d-a548-e5ac78b5f647) Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.189404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:13.190886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 04:26:13.192321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-be60d639-6806-4e3d-b4f0-3251603a9b0a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] Returning 404 to user: Volume 246fdf5f-6978-4e44-ab82-fe71297a8b18 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-be60d639-6806-4e3d-b4f0-3251603a9b0a tempest-VolumesSnapshotsTestJSON-1741608810 tempest-VolumesSnapshotsTestJSON-1741608810-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18" status: 404 len: 109 microversion: 2.1 time: 0.037949 Sep 27 04:26:13.193636 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 360/706] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:13 2022] GET /compute/v2.1/os-volumes/246fdf5f-6978-4e44-ab82-fe71297a8b18 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:26:13.967587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:13.969381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:13.994661 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:13.995860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:13.996632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:13.998021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:13.999885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:14.000903 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:14.164859 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:14.167835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:UPDATE=5,nova_cell1:SELECT=92,nova_cell1:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:14.175696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:14.177147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:14.178353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-71cf00e2-2a5a-4d2f-ad56-97cb0fcb4f93 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.220225 Sep 27 04:26:14.178901 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 347/707] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:13 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:14.479469 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:14.590624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad2c8bbb-6424-4276-9d86-d6b2da747c48 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Action: 'create', calling method: .version_select at 0x7fce707d7430>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-968716298", "metadata": {"Type": "work"}}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad2c8bbb-6424-4276-9d86-d6b2da747c48 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 121 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:14.599489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-968716298", "metadata": {"Type": "work"}}} Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:14.601205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-968716298', 'metadata': {'Type': 'work'}}}} Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise error Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi '#$%' Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.602145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 04:26:14.603521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 04:26:14.604698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 04:26:14.604698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.604698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ad2c8bbb-6424-4276-9d86-d6b2da747c48 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.604698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ad2c8bbb-6424-4276-9d86-d6b2da747c48 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.122539 Sep 27 04:26:14.604698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 361/708] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 04:26:14 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 124 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:26:14.610822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6439cd34-9d6c-4098-85c8-121b069806cd tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Action: 'create', calling method: .version_select at 0x7fce708cc160>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1021146690", "metadata": {"Type": "work"}}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6439cd34-9d6c-4098-85c8-121b069806cd tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 120 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:14.615696 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1021146690", "metadata": {"Type": "work"}}} Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: Sep 27 04:26:14.617181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1021146690', 'metadata': {'Type': 'work'}}}} Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise error Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi '0' Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.618116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 04:26:14.619905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 04:26:14.620987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 04:26:14.620987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.620987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6439cd34-9d6c-4098-85c8-121b069806cd tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.620987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6439cd34-9d6c-4098-85c8-121b069806cd tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010402 Sep 27 04:26:14.620987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 348/709] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 04:26:14 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:26:14.623195 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=7 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:14.627927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b324dcb5-3f02-4ebb-9943-99fa49ebee6d tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Action: 'create', calling method: .version_select at 0x7fce707d70d0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1276156631", "metadata": {"Type": "work"}}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b324dcb5-3f02-4ebb-9943-99fa49ebee6d tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 119 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:14.631511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1276156631", "metadata": {"Type": "work"}}} Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:14.633017 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1276156631', 'metadata': {'Type': 'work'}}}} Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise error Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi '' Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.633946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 04:26:14.635811 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 04:26:14.637214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 04:26:14.637214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.637214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b324dcb5-3f02-4ebb-9943-99fa49ebee6d tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.637214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b324dcb5-3f02-4ebb-9943-99fa49ebee6d tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010273 Sep 27 04:26:14.637214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 362/710] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 04:26:14 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:26:14.642628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9574bc31-d904-4042-ad63-fa42cdc0fbf7 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Calling method '.version_select at 0x7fce708e0790>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:14.671551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-9574bc31-d904-4042-ad63-fa42cdc0fbf7 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] HTTP exception thrown: Volume tempest-invalid-1634478082 could not be found. Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9574bc31-d904-4042-ad63-fa42cdc0fbf7 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1634478082 HTTP/1.1 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:14.673522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Volume tempest-invalid-1634478082 could not be found. Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'tempest-invalid-1634478082'} Sep 27 04:26:14.675109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1634478082 could not be found. (HTTP 404) (Request-ID: req-f0b70d2c-4b1f-45b6-a7b3-07aa7eefddad) Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.675956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:14.677171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1634478082 could not be found. Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:14.678367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1634478082 could not be found. Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1634478082 could not be found. Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-9574bc31-d904-4042-ad63-fa42cdc0fbf7 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 404 to user: Volume tempest-invalid-1634478082 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-9574bc31-d904-4042-ad63-fa42cdc0fbf7 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1634478082" status: 404 len: 99 microversion: 2.1 time: 0.036239 Sep 27 04:26:14.679571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 349/711] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:26:14 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1634478082 => generated 99 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 04:26:14.686268 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d59db169-686e-43f9-99ce-93d524df0281 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004637 Sep 27 04:26:14.686757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 363/712] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 04:26:14 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 04:26:14.694830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0a19df18-dcaf-4f64-96be-aaff63cd3658 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003672 Sep 27 04:26:14.695308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 350/713] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 04:26:14 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 04:26:14.703825 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-048211d2-53b7-4816-9c6a-fb9691accf1b tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Calling method '.version_select at 0x7fce709ceb80>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:14.731426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-048211d2-53b7-4816-9c6a-fb9691accf1b tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] HTTP exception thrown: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-048211d2-53b7-4816-9c6a-fb9691accf1b tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/os-volumes/3998b565-eb90-430a-b1fe-0dcca7106bc9 HTTP/1.1 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:14.733324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '3998b565-eb90-430a-b1fe-0dcca7106bc9'} Sep 27 04:26:14.734914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. (HTTP 404) (Request-ID: req-2513394d-59c0-4a13-bebb-00368cfcf682) Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.735868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 04:26:14.737211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 04:26:14.738610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-048211d2-53b7-4816-9c6a-fb9691accf1b tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 404 to user: Volume 3998b565-eb90-430a-b1fe-0dcca7106bc9 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-048211d2-53b7-4816-9c6a-fb9691accf1b tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/3998b565-eb90-430a-b1fe-0dcca7106bc9" status: 404 len: 109 microversion: 2.1 time: 0.034444 Sep 27 04:26:14.739882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 364/714] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:26:14 2022] DELETE /compute/v2.1/os-volumes/3998b565-eb90-430a-b1fe-0dcca7106bc9 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:14.744874 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d2348ed3-c54b-4f90-99c0-98e91838af68 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Calling method '.version_select at 0x7fce70985700>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:14.775484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-d2348ed3-c54b-4f90-99c0-98e91838af68 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] HTTP exception thrown: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d2348ed3-c54b-4f90-99c0-98e91838af68 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Request method failure captured: Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/os-volumes/409e8da5-ad8e-4473-81b0-575d9fb62c1e HTTP/1.1 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMntmAhKSShYF-7aHq0C8uAKctKMCW5x_bSVR6ZzB5DhKF_9HNBWC0fWhEeljr8nVW3V5Lv_tcuHHAR5nXyNgpLnU_-M4LierXN_t4SwaItlLZH6odpVHWa8b_4QbAQIUxbX9DsXFd6-RG2Y_lRf9VXxy_mrIYgQ2D-9bzaFaoM0xuhY Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:14.777500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_49ef5f872e954a9295f1d2e1a00632b8", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/49ef5f872e954a9295f1d2e1a00632b8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 49ef5f872e954a9295f1d2e1a00632b8 Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesNegativeTest-1525162857 Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: a293bc416349457ea83527a1ef42511f Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesNegativeTest-1525162857-project Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '409e8da5-ad8e-4473-81b0-575d9fb62c1e'} Sep 27 04:26:14.779040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. (HTTP 404) (Request-ID: req-5b1780f0-ce84-4129-98b3-d45c0a2db40f) Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.779981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:26:14.781321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 04:26:14.782688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d2348ed3-c54b-4f90-99c0-98e91838af68 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] Returning 404 to user: Volume 409e8da5-ad8e-4473-81b0-575d9fb62c1e could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d2348ed3-c54b-4f90-99c0-98e91838af68 tempest-VolumesNegativeTest-1525162857 tempest-VolumesNegativeTest-1525162857-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/409e8da5-ad8e-4473-81b0-575d9fb62c1e" status: 404 len: 109 microversion: 2.1 time: 0.038331 Sep 27 04:26:14.784036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 351/715] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:26:14 2022] GET /compute/v2.1/os-volumes/409e8da5-ad8e-4473-81b0-575d9fb62c1e => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:15.112082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:15.114246 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:15.121063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.121555 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.122035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.122733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.123190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.123624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.160091 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-38443a5d-ec12-401c-8edf-2ae4e27f2e77 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053292 Sep 27 04:26:15.160828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 365/716] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:26:15 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:15.177110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:15.178681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:15.189933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.191021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.192550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.193780 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.194380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.195227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.205203 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:15.206312 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:15.217200 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.218152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.218707 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.221869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:15.222461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:15.223132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:15.247399 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-55b7bd2d-8ab4-450d-b2bc-3c4413904f8e tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981" status: 200 len: 194 microversion: 2.1 time: 0.078689 Sep 27 04:26:15.248479 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 352/717] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 04:26:15 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981 => generated 194 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:15.407611 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:15.425688 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:15.427524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:15.431132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a5711b7f-cab4-4145-9757-c8db5ad05e0b tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.231006 Sep 27 04:26:15.443520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 366/718] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:15 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:16.471246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:16.471867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:16.478264 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:16.478759 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:16.479205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:16.479898 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:16.480372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:16.480797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:16.670142 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:16.689021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:16.690865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:16.692834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-405d4b02-c5e4-412c-94bd-927f23adfbc6 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.225253 Sep 27 04:26:16.693547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 353/719] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:16 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:16.702486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7400ba97-a1be-4adf-ad77-4fae773028c3"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:16.706476 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:16.715566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:16.716128 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:16.716776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:16.717486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:16.717967 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:16.718390 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:16.984063 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d426a9aa-96ac-4ad4-b43f-ee7e89f37756 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "POST /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.290083 Sep 27 04:26:16.984975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 367/720] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:26:16 2022] POST /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 194 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:17.516945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "name": "network-vif-plugged", "status": "completed", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:17.528229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:17.528914 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:17.529514 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:17.545301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] Creating event network-vif-plugged:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:26:17.554515 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-6ede03dd-d293-46e3-ad57-53d5ae99e417 req-11c64f52-3758-47d9-91f1-2a21322d161d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041908 Sep 27 04:26:17.556030 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 354/721] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:26:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:17.721952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:17.722648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:17.728842 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:17.729403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:17.729871 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:17.730561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:17.731011 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:17.731461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:17.866089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:17.875060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:17.876659 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:17.877942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-44242ba6-cd13-4549-a3b6-314ec3922525 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.160926 Sep 27 04:26:17.878582 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 368/722] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:17 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:18.905332 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:18.906007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:18.912063 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:18.912546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:18.913228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:18.913954 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:18.914416 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:18.914867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:19.056976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:19.065966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:19.069772 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:19.070982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7c05dca4-b808-42f2-a064-7a30c4dc2ac3 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.171045 Sep 27 04:26:19.073554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 355/723] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:18 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:20.100432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:20.101587 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:20.109590 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:20.110992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:20.111452 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:20.112158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:20.112782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:20.113206 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:20.252029 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:20.261422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:20.262987 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:20.264268 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ea6945ab-20ac-4919-8c5c-c4f661f0ffec tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1727 microversion: 2.20 time: 0.169033 Sep 27 04:26:20.265135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 369/724] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:20 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:21.292384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:21.293001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:21.295786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=58 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:21.300945 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:21.301382 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:21.301855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:21.302554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:21.303216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:21.304046 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:21.436737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:21.444712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:21.446202 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:21.447417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d7df1564-0826-4d73-ad01-298c815f3dba tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1714 microversion: 2.20 time: 0.159710 Sep 27 04:26:21.447962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 356/725] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:21 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1714 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:26:25.613621 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:25.615998 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:25.622519 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.623008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.623449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.624139 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.625152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.625583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.656855 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8e8e970d-84e3-4a50-b71c-3be9c9f2ee23 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.047115 Sep 27 04:26:25.657412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 370/726] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:26:25 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 371 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:25.800279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:25.800799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:25.807258 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.807739 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.808171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.808854 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.809308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.809731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.837612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-101704ee-fe2e-4273-a698-7c6b3c4a1d01 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981" status: 200 len: 194 microversion: 2.1 time: 0.041931 Sep 27 04:26:25.838207 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 357/727] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 04:26:25 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:25.848446 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:25.848945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:25.854759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.855246 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.855704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.856396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:25.856851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:25.857282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:25.935539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'services' on Instance uuid 710d8936-179a-412b-9f1d-148e6648cebe {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:26:26.006648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5d0c854e-c9f0-40ec-bc26-7d0b328f548a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981" status: 202 len: 0 microversion: 2.1 time: 0.161938 Sep 27 04:26:26.007227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 371/728] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:26:25 2022] DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/b41bbf3e-fd9e-4f10-883b-904118a15981 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:26:30.295268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:30.295754 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:30.302216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:30.302701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:30.303138 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:30.303818 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:30.304275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:30.304699 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:30.333002 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-51819f68-b9fb-48f1-88c8-2a57d9b61dd7 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/7400ba97-a1be-4adf-ad77-4fae773028c3" status: 200 len: 194 microversion: 2.1 time: 0.041709 Sep 27 04:26:30.333556 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 358/729] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 04:26:30 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/7400ba97-a1be-4adf-ad77-4fae773028c3 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:30.345040 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:30.345660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:30.352325 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:30.352809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:30.353247 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:30.353967 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:30.354425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:30.354856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:30.435239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'services' on Instance uuid 710d8936-179a-412b-9f1d-148e6648cebe {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:26:30.505189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b7a5991a-c4e6-496b-8194-de6e188f0c6b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/7400ba97-a1be-4adf-ad77-4fae773028c3" status: 202 len: 0 microversion: 2.1 time: 0.164767 Sep 27 04:26:30.505986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 372/730] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:26:30 2022] DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments/7400ba97-a1be-4adf-ad77-4fae773028c3 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:26:31.694118 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:31.696346 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:31.702731 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.703236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.703674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.704852 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.705308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.705732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.735449 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-55d9f7f3-4c35-47ae-a1ed-ab5a80950bd9 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045241 Sep 27 04:26:31.736134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 359/731] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:26:31 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:26:31.852787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:31.855057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:31.861083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.861559 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.862015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.862697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.863244 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.863981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.892511 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c6a71bae-d164-4f16-b8bb-83bf56340aad tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044543 Sep 27 04:26:31.893049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 373/732] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:26:31 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:26:31.955536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:31.956034 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:31.961957 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.962455 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.962903 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.963599 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:31.964060 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:31.964491 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:31.989037 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:26:32.062075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-82c365c4-39d8-443d-9b56-512b715050ef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 204 len: 0 microversion: 2.1 time: 0.110508 Sep 27 04:26:32.062649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 360/733] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:26:31 2022] DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:26:32.072410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:32.073013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:32.079568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:32.080049 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:32.080497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:32.081168 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:32.081619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:32.082060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:32.214482 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:32.222179 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-90fb639a-3c24-4902-86d3-fc60c3f06109 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1673 microversion: 2.1 time: 0.153795 Sep 27 04:26:32.222734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 374/734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:32 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:33.246145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:33.246850 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:33.253082 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:33.253564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:33.254035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:33.254715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:33.255170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:33.255594 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:33.384649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:33.392847 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-210b04cf-1434-4fa8-80a1-314d8ae1a71d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1673 microversion: 2.1 time: 0.150932 Sep 27 04:26:33.393405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 361/735] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:33 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1673 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:34.416024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:34.416766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:34.423876 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:34.424360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:34.424833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:34.425521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:34.425995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:34.426417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:34.556669 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:34.564249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-71c2fe1f-0240-43f7-891a-d432c836b21b tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1673 microversion: 2.1 time: 0.152319 Sep 27 04:26:34.564807 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 375/736] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:34 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1673 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:35.587089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:35.587746 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:35.594933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:35.595629 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:35.596068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:35.596975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:35.597434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:35.597886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:35.729361 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:35.738673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a50fa669-aaf3-48a6-958a-2b9e5114aa70 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1673 microversion: 2.1 time: 0.155318 Sep 27 04:26:35.739610 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 362/737] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:35 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1673 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:36.761089 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "710d8936-179a-412b-9f1d-148e6648cebe", "name": "network-vif-deleted", "tag": "4e861415-84ce-4923-aa54-770c8249a620"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:26:36.767302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:36.768720 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:36.782557 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:36.785464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:36.787726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:36.789476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:36.790932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:36.791757 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:36.792291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:36.794100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:36.794630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:36.819351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] Creating event network-vif-deleted:4e861415-84ce-4923-aa54-770c8249a620 for instance 710d8936-179a-412b-9f1d-148e6648cebe on hv-cinder-85918 Sep 27 04:26:36.829435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-cfdda2cd-4236-42e3-884c-2aa3e5d731cd req-8e38b8b1-1165-4843-9d09-47e5b5a991f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.266652 Sep 27 04:26:36.831713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 376/738] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:26:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:26:36.864420 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:36.864688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:36.865930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:36.868309 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lazy-loading 'fault' on Instance uuid 710d8936-179a-412b-9f1d-148e6648cebe {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:26:36.918771 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f3438d850ce541cb9027ea3d5fe7dbf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c6cc70cbd2b4c04864f33ebafa06b5a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:26:36.927406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-584c28fb-2bb5-49e5-99bb-4a3d4c3ad71a tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 200 len: 1327 microversion: 2.1 time: 0.164756 Sep 27 04:26:36.929911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 363/739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:36 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 1327 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:37.952925 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:37.953545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:37.959628 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:37.960104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:37.960537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:37.961214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:37.961662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:37.962110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:37.974852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe HTTP/1.1 Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:37.976420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:37.978013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '710d8936-179a-412b-9f1d-148e6648cebe'} Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:37.979016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:37.980455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:37.981695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:37.981695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e3bca2a5-8bc4-447a-a74e-e06fd5bc2fbb tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 404 len: 111 microversion: 2.1 time: 0.029687 Sep 27 04:26:37.981695 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 377/740] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:37 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:37.986846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-da33a627-fcfd-4a27-94c3-ba44cb951d7d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '.version_select at 0x7fce70952820>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:38.006473 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-da33a627-fcfd-4a27-94c3-ba44cb951d7d tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-739774848" status: 202 len: 0 microversion: 2.1 time: 0.023804 Sep 27 04:26:38.007433 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 364/741] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:26:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-739774848 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:26:41.927746 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:41.928277 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:41.934759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:41.935237 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:41.935666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:41.936342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:41.936790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:41.937213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:41.950098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe HTTP/1.1 Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:41.951638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.953296 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '710d8936-179a-412b-9f1d-148e6648cebe'} Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:41.954407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.955988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.957653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:41.957653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1d107f82-2e65-4a44-a30b-3c787951caef tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 404 len: 111 microversion: 2.1 time: 0.029051 Sep 27 04:26:41.957653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 378/742] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:26:41 2022] DELETE /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:26:41.962114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:41.962666 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: 710d8936-179a-412b-9f1d-148e6648cebe] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:41.968970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:41.969984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:41.970419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:41.971094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:41.971547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:41.971970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:41.984834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe HTTP/1.1 Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:41.987065 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.988625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '710d8936-179a-412b-9f1d-148e6648cebe'} Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:41.989644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. Sep 27 04:26:41.991232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:41.992665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance 710d8936-179a-412b-9f1d-148e6648cebe could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:41.992665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0df83816-f099-43d1-9e47-67d44b976173 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe" status: 404 len: 111 microversion: 2.1 time: 0.031093 Sep 27 04:26:41.992665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 365/743] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:41 2022] GET /compute/v2.1/servers/710d8936-179a-412b-9f1d-148e6648cebe => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:43.127999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:43.128516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:43.134801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:43.135286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:43.135726 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:43.136408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:43.136864 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:43.137316 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:43.152798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec HTTP/1.1 Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:26:43.154347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.155900 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'e24dba8d-ecd7-482f-9c93-79e714785fec'} Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:43.156713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.158780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.160276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:43.160276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-aa66fcc3-cff3-475a-ab02-7eab686f6389 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 404 len: 111 microversion: 2.1 time: 0.031745 Sep 27 04:26:43.160276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 379/744] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:26:43 2022] DELETE /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:26:43.165040 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:43.165685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] [instance: e24dba8d-ecd7-482f-9c93-79e714785fec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:43.172005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:43.172864 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:43.172991 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:43.173710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:43.177151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:43.177151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:43.188342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] HTTP exception thrown: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Request method failure captured: Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec HTTP/1.1 Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn0ENFeImLyCZzHA0KP77iEuNWCb-YR4UDskvCW5ao_pC3dqqt2jDWlC55udt-PpUigfx-OtOSAdMNrSEORfxzmBjHnNfaoYfAJWTtnaQqsBe3VBtUhsFtL3yZr_QL0c6hXtpFIKBITugPtX-DvrjyiPoQSH6xnevUWiefn1synwjQ Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:26:43.189920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c6cc70cbd2b4c04864f33ebafa06b5a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c6cc70cbd2b4c04864f33ebafa06b5a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 1c6cc70cbd2b4c04864f33ebafa06b5a Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1830949147 Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: f3438d850ce541cb9027ea3d5fe7dbf3 Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeTestJSON-1830949147-project Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.192297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'e24dba8d-ecd7-482f-9c93-79e714785fec'} Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:26:43.193714 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. Sep 27 04:26:43.195365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:26:43.196682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] Returning 404 to user: Instance e24dba8d-ecd7-482f-9c93-79e714785fec could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:26:43.196682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f0f06339-977d-4d2c-88b5-620c8c78f537 tempest-AttachVolumeTestJSON-1830949147 tempest-AttachVolumeTestJSON-1830949147-project] 10.222.0.26 "GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec" status: 404 len: 111 microversion: 2.1 time: 0.030314 Sep 27 04:26:43.196682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 366/745] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:26:43 2022] GET /compute/v2.1/servers/e24dba8d-ecd7-482f-9c93-79e714785fec => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:26:59.745942 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:59.749477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:59.754699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=30 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:59.755396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:59.762194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.762686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.763125 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:59.763822 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.764378 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.764803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:59.795793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b7fe01a4-f11c-4f53-aeb6-36eaa20f6af5 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054066 Sep 27 04:26:59.796348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 380/746] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 04:26:59 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:26:59.859908 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:59.860403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:59.863749 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:59.864632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:26:59.870524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.871021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.871467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:59.872192 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.872657 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.873089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:59.899828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:26:59.978675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-436862aa-6636-4de6-ac0b-9a49d3d8440e tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 204 len: 0 microversion: 2.20 time: 0.122497 Sep 27 04:26:59.979313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 367/747] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 04:26:59 2022] DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 04:26:59.988308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:26:59.988921 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:26:59.994937 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.995433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.995881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:26:59.996582 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:26:59.997043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:26:59.997473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:00.130186 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:00.138157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:00.139700 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:00.140914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-095bf0e9-7be3-43e8-8740-99e5c2d3a870 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.157200 Sep 27 04:27:00.141480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 381/748] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:26:59 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:27:01.167263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:01.168067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:01.174310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:01.174808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:01.175255 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:01.175944 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:01.176406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:01.176834 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:01.318752 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:01.326797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:01.328241 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:01.329446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4f9abeab-543f-422c-9b4d-febb1ce8988f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1720 microversion: 2.20 time: 0.166697 Sep 27 04:27:01.330001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 368/749] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:27:01 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1720 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:27:02.318512 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c6879a2-4ef5-4e16-8066-a2f05eca633c", "name": "network-vif-deleted", "tag": "b5c073b9-7000-4407-b92c-8e4279bf0f72"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:02.334551 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:02.335032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:02.335673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:02.351204 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] Creating event network-vif-deleted:b5c073b9-7000-4407-b92c-8e4279bf0f72 for instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c on hv-cinder-85918 Sep 27 04:27:02.355564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-03916809-bb0e-41b0-9b57-6401c0b81d02 req-b67aaffb-94c1-41aa-9510-3ea984be227f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041590 Sep 27 04:27:02.356362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 382/750] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:27:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:27:02.359985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:02.362151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:02.373484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:02.377518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:02.378365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:02.379956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:02.380655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:02.381290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:02.485755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:02.497106 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:02.499210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '7c56e7333ce4450b8cbd5caeafc73947', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312fa89a4d9b43c5919b7bad5411002a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:02.500521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-543fbba4-803a-4cfe-9824-5f0111e0de1f tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 200 len: 1366 microversion: 2.20 time: 0.147716 Sep 27 04:27:02.501482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 369/751] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:27:02 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 04:27:03.527402 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:03.528091 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:03.534187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:03.534665 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:03.535100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:03.535782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:03.536231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:03.536653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:03.549814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c HTTP/1.1 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:27:03.551391 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:03.553078 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '5c6879a2-4ef5-4e16-8066-a2f05eca633c'} Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:27:03.554042 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:03.555661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:03.557188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:27:03.557188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-44ca0c11-09bf-445c-9f09-f96f2cae6880 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 404 len: 111 microversion: 2.20 time: 0.030067 Sep 27 04:27:03.557188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 383/752] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:27:03 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:27:03.561786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2fbcbbdb-5338-4dd6-88ca-22fa40de9e94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '.version_select at 0x7fce708b2f70>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:03.581228 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2fbcbbdb-5338-4dd6-88ca-22fa40de9e94 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1623599026" status: 204 len: 0 microversion: 2.20 time: 0.022593 Sep 27 04:27:03.581785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 370/753] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 04:27:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1623599026 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 04:27:05.351018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:05.351604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:05.359000 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:05.359492 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:05.359932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:05.360632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:05.361083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:05.361766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:05.377577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c HTTP/1.1 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:27:05.379311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.380803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '5c6879a2-4ef5-4e16-8066-a2f05eca633c'} Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:27:05.381787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.382972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.384231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:27:05.384231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3e7b325e-dbef-442c-9afc-82b8089c06bd tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 404 len: 111 microversion: 2.20 time: 0.034827 Sep 27 04:27:05.384231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 384/754] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 04:27:05 2022] DELETE /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:27:05.394723 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:05.395360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 5c6879a2-4ef5-4e16-8066-a2f05eca633c] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:05.402767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:05.403470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:05.403919 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:05.404670 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:05.405133 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:05.405595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:05.420278 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c HTTP/1.1 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:27:05.422042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.423673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '5c6879a2-4ef5-4e16-8066-a2f05eca633c'} Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:27:05.425010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. Sep 27 04:27:05.426250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:05.427320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 5c6879a2-4ef5-4e16-8066-a2f05eca633c could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:27:05.427320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7fd02e55-29e5-4429-90a1-4a0944846147 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c" status: 404 len: 111 microversion: 2.20 time: 0.033983 Sep 27 04:27:05.427320 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 371/755] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:27:05 2022] GET /compute/v2.1/servers/5c6879a2-4ef5-4e16-8066-a2f05eca633c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:27:06.565830 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:06.566327 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:06.572603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:06.573084 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:06.573513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:06.574329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:06.574782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:06.575203 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:06.590603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 HTTP/1.1 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:27:06.592153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.593776 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '333a79d8-9886-4cc6-bbd4-335bee1d98b6'} Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:27:06.594566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.596412 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.597595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:27:06.597595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1af6a7b6-31fb-48ee-9ca6-a40174e2d920 tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 404 len: 111 microversion: 2.20 time: 0.031580 Sep 27 04:27:06.597595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 385/756] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 04:27:06 2022] DELETE /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:27:06.603174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:06.603782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] [instance: 333a79d8-9886-4cc6-bbd4-335bee1d98b6] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:06.610888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:06.611378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:06.612413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:06.613104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:06.613563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:06.614085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:06.627342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] HTTP exception thrown: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Request method failure captured: Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 HTTP/1.1 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnn5hyyY9J_2qhcxjGdFzin1vneAMAdDWJM3-mBj_d6xsexchg4HyRhM0Pmrk36_tggR1UMA1Rgad8P9D00fjrQ5HjrIwIq1G-syauQtXHdQ-IcxousxTsAvL5loEIhw_Fv1N1WQoSY4PnQcqksfavytojiVRqJYkCAtb7f7od7bj0Q Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:27:06.628717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_312fa89a4d9b43c5919b7bad5411002a", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/312fa89a4d9b43c5919b7bad5411002a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 312fa89a4d9b43c5919b7bad5411002a Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1565737032 Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 7c56e7333ce4450b8cbd5caeafc73947 Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1565737032-project Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.630583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '333a79d8-9886-4cc6-bbd4-335bee1d98b6'} Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:27:06.631597 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. Sep 27 04:27:06.633079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:27:06.634293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] Returning 404 to user: Instance 333a79d8-9886-4cc6-bbd4-335bee1d98b6 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:27:06.634293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-09925be1-6ba4-4c76-b1a2-0cdd66ebe6bf tempest-AttachVolumeShelveTestJSON-1565737032 tempest-AttachVolumeShelveTestJSON-1565737032-project] 10.222.0.26 "GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6" status: 404 len: 111 microversion: 2.20 time: 0.030924 Sep 27 04:27:06.634293 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 372/757] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 04:27:06 2022] GET /compute/v2.1/servers/333a79d8-9886-4cc6-bbd4-335bee1d98b6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:27:15.153497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1da73e2f-1c18-4de0-977b-c7190742995e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: .version_select at 0x7fce7084c0d0>, body: {"keypair": {"name": "tempest-keypair-516491899", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKiEtYO/Co82P6hbqIA+0SOxai/97C/vXF+myyqiJckjRpl3kmkOa0ylgxh9lrJMBL6zp2tWCZm+HHaY46MSKGZWO5eXC9ZsQzATPHL8XDItGhXbFCiTdeWvdTiJc3Auog=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:15.169662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-1da73e2f-1c18-4de0-977b-c7190742995e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:27:15.173536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-1da73e2f-1c18-4de0-977b-c7190742995e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:27:15.199108 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1da73e2f-1c18-4de0-977b-c7190742995e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.158036 Sep 27 04:27:15.199710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 386/758] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:27:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:27:16.607737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1733808371", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cba0fe06-d029-4450-9dfd-fb0123caa50b"}], "security_groups": [{"name": "tempest-securitygroup--1406019748"}], "key_name": "tempest-keypair-516491899", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:16.617017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:16.619104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:16.715489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] validate_networks() for [('cba0fe06-d029-4450-9dfd-fb0123caa50b', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:27:17.146057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:27:17.146552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:27:17.147080 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:27:17.147503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:27:17.147925 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:27:17.156786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:17.157398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:17.158018 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:17.158875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:17.159393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:17.160026 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:17.176984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:27:17.180878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:27:17.193592 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:27:17.195921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:27:17.250357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c2aa3088-e7a8-4cd8-a193-1d2a07d5dc94 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.646767 Sep 27 04:27:17.250999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 373/759] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:27:16 2022] POST /compute/v2.1/servers => generated 402 bytes in 648 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:27:17.263537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7b331a4f-9535-44d1-a7e4-ea07dc660037 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:17.264143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7b331a4f-9535-44d1-a7e4-ea07dc660037 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:17.268991 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=20 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:17.328518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7b331a4f-9535-44d1-a7e4-ea07dc660037 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:17.337813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7b331a4f-9535-44d1-a7e4-ea07dc660037 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1241 microversion: 2.1 time: 0.078824 Sep 27 04:27:17.338480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 387/760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:17 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1241 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:27:18.362645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:18.363406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:18.369998 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:18.370793 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:18.371309 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:18.372093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:18.372743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:18.373390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:18.436893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:18.444804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-96cf9aed-0f34-47a5-ae49-59c9ab2b1691 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1245 microversion: 2.1 time: 0.086298 Sep 27 04:27:18.445373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 374/761] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:18 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1245 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:19.475680 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:19.476293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:19.482996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:19.483483 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:19.483917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:19.484605 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:19.485052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:19.485467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:19.550432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:19.558637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-87411729-9921-484d-b96d-887173970e6d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1299 microversion: 2.1 time: 0.086714 Sep 27 04:27:19.559415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 388/762] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:19 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1299 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:20.587427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:20.588294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:20.595487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:20.596204 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:20.596960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:20.597730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:20.598339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:20.599025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:20.801869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:20.811163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-54201c49-4cc2-4af6-84f4-9db1bc26b143 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.231419 Sep 27 04:27:20.812253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 375/763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:20 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:20.862550 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "93c1b78a-1f68-4d8b-a00a-15ed4dfe4779", "tag": "59621d87-e6fa-445c-98d8-2f933c046f0d"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:20.882353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:20.884215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:20.885113 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:20.908929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] Creating event network-changed:59621d87-e6fa-445c-98d8-2f933c046f0d for instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 on hv-cinder-85918 Sep 27 04:27:20.913878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-f2e0c399-2435-47c0-8854-daac14bfd8a9 req-6bcd0524-6cec-4f56-827a-aa9e43a26914 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055213 Sep 27 04:27:20.914828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 389/764] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:27:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:27:21.835507 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:21.836201 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:21.842492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:21.842960 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:21.843397 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:21.844073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:21.844520 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:21.844974 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:21.980494 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:21.988843 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-41717d75-1fed-465f-8f32-158fc6f27dd9 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.157122 Sep 27 04:27:21.989337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 376/765] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:21 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:23.013634 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:23.014370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:23.021101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:23.021591 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:23.022059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:23.022754 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:23.023211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:23.023640 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:23.147970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:23.155387 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fdfd876e-014e-489d-99ec-50023d932dbe tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.146442 Sep 27 04:27:23.155944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 390/766] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:23 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:24.177402 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:24.177956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:24.184604 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:24.185101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:24.185538 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:24.186274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:24.186730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:24.187206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:24.323858 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:24.331962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ea8dbc1b-95d8-4a68-9b50-ee91fb265e7b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.158467 Sep 27 04:27:24.332521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 377/767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:24 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:25.353800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:25.354417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:25.360959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:25.362114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:25.362568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:25.363259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:25.363713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:25.364143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:25.648929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:25.656641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-585b3cbf-493f-42c6-afbe-b40621c0b9fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.306844 Sep 27 04:27:25.657206 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 391/768] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:25 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:26.680417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:26.681051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:26.687534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:26.688074 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:26.688502 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:26.689211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:26.689671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:26.692257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:26.824007 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:26.831814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-627f2070-98fd-4203-8f01-ba0601c8be56 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.155242 Sep 27 04:27:26.846448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 378/769] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:26 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:26.846777 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93c1b78a-1f68-4d8b-a00a-15ed4dfe4779", "name": "network-vif-plugged", "status": "completed", "tag": "59621d87-e6fa-445c-98d8-2f933c046f0d"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:26.852161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:26.852759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:26.853160 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:26.865881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] Creating event network-vif-plugged:59621d87-e6fa-445c-98d8-2f933c046f0d for instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 on hv-cinder-85918 Sep 27 04:27:26.871173 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-a471548a-f9b3-48ad-b37e-beab8ad63784 req-f7b7856c-52d8-46c0-b973-cb96367b794e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033374 Sep 27 04:27:26.871920 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 392/770] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:27:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:27:27.856145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:27.856764 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:27.858603 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:27.859263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:27.865573 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:27.866102 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:27.866573 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:27.867276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:27.867738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:27.868166 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:27.998842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:28.006810 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f41ed5db-d54b-44bc-98ee-3b8d388b577e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.154551 Sep 27 04:27:28.007367 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 379/771] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:27 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:29.030317 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:29.031006 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:29.032904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:29.039323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:29.039801 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:29.040497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:29.041173 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:29.041623 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:29.042071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:29.167567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:29.175508 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9b936088-d17c-49f7-ae01-2ab4bf382d62 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1367 microversion: 2.1 time: 0.149689 Sep 27 04:27:29.176055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 393/772] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:29 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:30.198992 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:30.199518 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:30.206459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:30.206955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:30.207401 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:30.208091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:30.208552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:30.208984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:30.338229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:30.346501 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5c156573-d90c-4578-ae5a-b13b78190a78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1556 microversion: 2.1 time: 0.150794 Sep 27 04:27:30.347058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 380/773] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:30 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1556 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:30.368588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:30.369241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:30.371593 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:30.376809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:30.377287 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:30.377716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:30.378420 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:30.378869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:30.379290 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:30.454781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4eb85055-3027-4054-94f1-11980bf4d5db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089995 Sep 27 04:27:30.456082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 394/774] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:27:30 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:27:31.653372 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "93c1b78a-1f68-4d8b-a00a-15ed4dfe4779", "tag": "59621d87-e6fa-445c-98d8-2f933c046f0d"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:31.664524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:31.665050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:31.665766 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:31.677682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] Creating event network-changed:59621d87-e6fa-445c-98d8-2f933c046f0d for instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 on hv-cinder-85918 Sep 27 04:27:31.683286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-4e0d4db4-72ef-4781-b632-b0fbd279e0d4 req-307ac7bd-c623-42d3-9f7c-2c924c549166 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033656 Sep 27 04:27:31.687417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 381/775] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:27:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:27:33.614013 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:33.614545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:33.620357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:33.620840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:33.621266 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:33.621982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:33.622432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:33.622848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:33.647110 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lazy-loading 'flavor' on Instance uuid 3063ca42-9212-4df2-af49-0d7d4a88c0ee {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:27:33.720753 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3ede8319-cf48-4b74-969b-f18e2a1099f8 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "POST /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee/action" status: 202 len: 0 microversion: 2.1 time: 0.111190 Sep 27 04:27:33.721704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 395/776] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:27:33 2022] POST /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:27:33.730733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:33.731445 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:33.737808 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:33.738314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:33.738764 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:33.739700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:33.740223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:33.740767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:33.876913 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:33.886315 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5c103fd0-2792-452b-b6ad-7c81bf5c1573 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.160043 Sep 27 04:27:33.887079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 382/777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:33 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:34.913219 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:34.913973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:34.920567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:34.921050 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:34.921488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:34.922212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:34.922670 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:34.923091 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:35.049794 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:35.057035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9bd8c8fb-242a-421e-84cb-c25a2fe84ec2 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.149463 Sep 27 04:27:35.057590 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 396/778] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:34 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:36.081174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:36.081701 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:36.088315 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:36.088831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:36.089267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:36.089981 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:36.090438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:36.090915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:36.217775 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:36.225910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3edc72da-0570-4951-85b6-781ebf050f95 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.148867 Sep 27 04:27:36.226471 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 383/779] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:36 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:37.248826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:37.249436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:37.256521 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:37.257003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:37.257440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:37.258156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:37.258609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:37.259036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:37.391979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:37.400321 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-64bb338a-d161-4dc4-9301-ac6b69a26fc7 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.155398 Sep 27 04:27:37.400879 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 397/780] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:37 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:38.424505 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:38.425877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:38.432243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:38.432730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:38.433172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:38.434009 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:38.434475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:38.434915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:38.570376 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:38.578017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fab023b4-c063-4e00-8fae-419f020f8e8e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.157596 Sep 27 04:27:38.578602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 384/781] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:38 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:39.601635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:39.602410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:39.608644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:39.609119 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:39.609548 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:39.610251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:39.610703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:39.611126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:39.742664 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:39.750267 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-093a7617-6256-4c85-bcf9-3b49179d3626 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.153053 Sep 27 04:27:39.750813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 398/782] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:39 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:40.772400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:40.773034 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:40.779349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:40.779838 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:40.780279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:40.780961 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:40.781427 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:40.781881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:40.913619 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:40.924999 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-64e00707-c682-4dd4-9b78-ad2990313e83 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.156380 Sep 27 04:27:40.926261 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 385/783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:40 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:41.949324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:41.950189 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:41.956713 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:41.957839 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:41.958435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:41.959143 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:41.959599 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:41.960126 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:42.096663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:42.104850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a0df04f6-192f-4902-9ab3-8cc1b6b5a816 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.159389 Sep 27 04:27:42.105428 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 399/784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:41 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:43.129153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:43.130017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:43.136612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:43.137107 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:43.137602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:43.138332 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:43.138982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:43.139411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:43.272032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:43.279811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4e936c32-89cf-43a4-b289-97630704fc7d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.154533 Sep 27 04:27:43.280377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 386/785] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:43 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:44.304070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:44.305158 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:44.312067 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:44.312620 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:44.313117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:44.313904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:44.314426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:44.314940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:44.455873 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:44.464381 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a43059e9-930b-450a-a51d-7b4a5ac6fb63 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.164963 Sep 27 04:27:44.464948 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 400/786] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:44 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:27:45.488755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:45.489380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:45.495371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:45.495863 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:45.496310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:45.497008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:45.497475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:45.497943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:45.638424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:45.647389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-61db4636-fed4-4ff6-83fd-c86b4c1d47d6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.163240 Sep 27 04:27:45.647983 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 387/787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:45 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:46.670880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:46.671494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:46.678057 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.678561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.678982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:46.679671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.680127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.680557 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:46.816077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:46.823767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-734882ae-0476-482d-89cf-6ab6daa49fa6 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1572 microversion: 2.1 time: 0.156931 Sep 27 04:27:46.824324 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 401/788] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:46 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1572 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:46.846726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:46.847962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:46.850441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=44,nova_api:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:46.856033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.856516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.856950 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:46.857654 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.858136 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.858665 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:46.863333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=83 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:46.888656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120424) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 04:27:46.953930 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4458d630-9fa9-48e9-817f-94383dd59a5a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "POST /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee/action" status: 202 len: 0 microversion: 2.1 time: 0.111431 Sep 27 04:27:46.954538 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 388/789] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 04:27:46 2022] POST /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:27:46.968245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:46.968245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:46.974969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.975461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.976015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:46.976916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:46.977454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:46.978104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:47.122882 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:47.131886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8a331cd0-45d1-4f07-bd38-a8a77ac35f09 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1580 microversion: 2.1 time: 0.172352 Sep 27 04:27:47.132466 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 402/790] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:46 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1580 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:48.155287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:48.156047 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:48.163316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:48.163949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:48.164508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:48.165305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:48.166152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:48.166681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:48.305043 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:48.312781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b3c5df1d-b44f-4f8a-8af1-68b464a36d65 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1522 microversion: 2.1 time: 0.161917 Sep 27 04:27:48.313360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 389/791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:48 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:49.338850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:49.339473 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:49.346732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:49.347233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:49.347711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:49.348423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:49.348894 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:49.349326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:49.366747 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:27:49.547382 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:49.555970 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5c754d12-6c53-4f82-b8a8-1e8a375f6f66 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.222760 Sep 27 04:27:49.556906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 403/792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:49 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:50.018334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "name": "network-vif-unplugged", "status": "completed", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:27:50.029401 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:50.030014 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:50.030525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:50.043222 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] Creating event network-vif-unplugged:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:27:50.048328 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-27fb6481-63de-405f-a203-c00e76823e6d req-5cacdf24-3a8c-4ee0-8d3b-341e620ab254 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034917 Sep 27 04:27:50.048946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 390/793] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:27:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:27:50.580358 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:50.580974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:50.587642 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:50.588129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:50.588567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:50.589323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:50.589797 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:50.590223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:50.724122 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:50.731927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b2086ca3-ba27-4d3e-a678-a1730e9b65e0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.155441 Sep 27 04:27:50.732677 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 404/794] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:50 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:51.757440 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:51.758176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:51.764317 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:51.764833 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:51.765273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:51.765979 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:51.766440 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:51.766865 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:51.899052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:51.906725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-540fee44-45cf-4567-8b6a-342315e85df0 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.154471 Sep 27 04:27:51.907288 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 391/795] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:51 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:52.930335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:52.931093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:52.937603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:52.938114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:52.938552 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:52.939238 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:52.939694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:52.940121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:53.065627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:53.073692 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3a710869-cb33-462c-adf6-7b928f576282 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.147352 Sep 27 04:27:53.074300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 405/796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:52 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:54.096408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:54.097016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:54.103813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:54.104301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:54.104738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:54.105468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:54.105956 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:54.106386 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:54.385677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:54.393337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2595eba6-0da2-4489-a9dd-6dca1ca8b42c tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.300823 Sep 27 04:27:54.393909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 392/797] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:54 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:55.416637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:55.417254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:55.423426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:55.423909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:55.424347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:55.425039 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:55.425534 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:55.425988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:55.560198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:55.568927 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4660c14d-f275-42e8-8115-8115d287c7bd tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.156261 Sep 27 04:27:55.569379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 406/798] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:55 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:56.593246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:56.593827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:56.600818 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:56.601301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:56.601737 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:56.602443 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:56.602907 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:56.603331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:56.731779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:56.739526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-36ebb311-1d9b-47db-a6a2-213e6e5520fa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1578 microversion: 2.1 time: 0.150082 Sep 27 04:27:56.740178 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 393/799] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:56 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1578 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:27:57.762980 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:27:57.763588 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:27:57.769944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:57.770426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:57.770861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:57.771538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:27:57.771989 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:27:57.772408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:27:57.898809 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:27:57.906361 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fa371761-9ff2-4f0e-ac6c-128ce4f4e2a5 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1565 microversion: 2.1 time: 0.147306 Sep 27 04:27:57.906915 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 407/800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:27:57 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1565 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:00.069297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "name": "network-vif-plugged", "status": "completed", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:00.079778 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:00.080291 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:00.080727 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:00.092130 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] Creating event network-vif-plugged:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:28:00.096966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-cad73960-d91d-471a-9b5e-4dbca7c79a23 req-cb3dcfc1-f541-4796-a8ad-a9c5f047b85f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031913 Sep 27 04:28:00.097595 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 394/801] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:28:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:28:07.824053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:07.824598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:07.827083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=27 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:07.832462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:07.833009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:07.833442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:07.834157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:07.834610 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:07.835033 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:07.857490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c18ead1e-16df-48bd-964c-369f8052032f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/ips" status: 200 len: 149 microversion: 2.1 time: 0.037313 Sep 27 04:28:07.858069 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 408/802] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:28:07 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/ips => generated 149 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:28:09.232628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "479a0dc4-bdbb-429d-8219-e3a3e5a11211"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:09.235330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:09.241799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:09.242292 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:09.242732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:09.243422 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:09.243920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:09.244352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:09.500830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-121012ca-e576-45ad-98db-a3152ef4b638 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.273423 Sep 27 04:28:09.501511 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 395/803] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:28:09 2022] POST /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments => generated 194 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:28:15.985848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-fef621ed-086b-4125-a44c-07eb1978c520 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: .version_select at 0x7fce70882d30>, body: {"keypair": {"name": "tempest-keypair-368356529", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBqzqIaznzNffU3TZ+rhoxITzvuSWk4SFBkMe2+wVmFOhN4P/kDvztHuVm0chmsprMcsDqy5slW02J90mQHMAAC9x3QphTwgSrafq0ON6onJk/dOiPZ3zc0D3koEDk80yw=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:16.001632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-fef621ed-086b-4125-a44c-07eb1978c520 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:28:16.005518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-fef621ed-086b-4125-a44c-07eb1978c520 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:28:16.027181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-fef621ed-086b-4125-a44c-07eb1978c520 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045162 Sep 27 04:28:16.027693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 409/804] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:28:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:28:17.265355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1206305845", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cba0fe06-d029-4450-9dfd-fb0123caa50b"}], "security_groups": [{"name": "tempest-securitygroup--1844079817"}], "key_name": "tempest-keypair-368356529", "user_data": "}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:17.275580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:17.369812 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] validate_networks() for [('cba0fe06-d029-4450-9dfd-fb0123caa50b', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:28:17.741650 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:28:17.742115 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:28:17.742589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:28:17.742996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:28:17.743405 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:28:17.751867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:17.752348 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:17.752788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:17.753498 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:17.753979 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:17.754402 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:17.761798 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=49,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:17.774530 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:28:17.778576 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:28:17.789275 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:28:17.791952 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:28:17.838862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e26b7ab6-cc20-4492-91ca-8f0fc3d054dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.578626 Sep 27 04:28:17.839776 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 396/805] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:28:17 2022] POST /compute/v2.1/servers => generated 402 bytes in 580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:28:17.850939 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-13dd13f5-d222-4f06-9860-a69f49adbf76 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:17.851560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-13dd13f5-d222-4f06-9860-a69f49adbf76 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:17.853607 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=33 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:17.906638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-13dd13f5-d222-4f06-9860-a69f49adbf76 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:17.913426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-13dd13f5-d222-4f06-9860-a69f49adbf76 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1241 microversion: 2.1 time: 0.066469 Sep 27 04:28:17.914024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 410/806] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:17 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1241 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:18.936817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:18.937358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:18.944705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:18.945290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:18.945726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:18.946747 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:18.947274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:18.947915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:19.031280 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:19.040575 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-acf1d47d-589b-4e4f-810d-82bc6552f39a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1311 microversion: 2.1 time: 0.108265 Sep 27 04:28:19.041179 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 397/807] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:18 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1311 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:20.065053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:20.065663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:20.072123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:20.072546 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:20.072981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:20.073679 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:20.074161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:20.074584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:20.211770 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:20.219505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5910c4e0-8f29-4dbf-8f39-f842ba4d3f8e tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.158382 Sep 27 04:28:20.220073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 411/808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:20 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:20.357521 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7842fec9-9ef4-47fb-b9ff-987b5ba3468b", "tag": "765ce5cc-318e-4b81-afac-627ffadfa36a"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:20.368561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:20.369048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:20.369516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:20.384016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] Creating event network-changed:765ce5cc-318e-4b81-afac-627ffadfa36a for instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b on hv-cinder-85918 Sep 27 04:28:20.388927 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-d1343156-e42b-410b-9d68-6757a298c582 req-91f2e57d-8b1b-4c15-98b0-d62d6ceb2f94 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043797 Sep 27 04:28:20.389516 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 398/809] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:28:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:28:21.244012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:21.244628 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:21.251132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:21.251617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:21.252054 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:21.252737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:21.253192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:21.253617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:21.382165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:21.389702 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e4731f9e-3b42-452b-ba55-9b9b3f64c475 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.149678 Sep 27 04:28:21.390258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 412/810] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:21 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:22.413144 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:22.413866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:22.420190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:22.420672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:22.421099 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:22.421792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:22.422248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:22.422667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:22.555487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:22.563095 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-96bd125f-6c0a-4ea1-9888-158e30014103 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.154641 Sep 27 04:28:22.563653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 399/811] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:22 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:23.586401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:23.587012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:23.593726 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:23.594242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:23.594681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:23.595373 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:23.595830 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:23.596258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:23.728117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:23.735714 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e927205d-b16c-4cac-a7e2-c913440423c0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.153757 Sep 27 04:28:23.736280 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 413/812] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:23 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:24.758848 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:24.759486 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:24.765850 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:24.766338 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:24.766777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:24.767463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:24.767923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:24.768388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:24.892679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:24.900267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-79d3ead4-c948-4f93-a6e9-67714c61f5ff tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.145816 Sep 27 04:28:24.900824 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 400/813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:24 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:25.923333 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:25.923954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:25.930298 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:25.930788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:25.931230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:25.931959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:25.932417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:25.932849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:26.065850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:26.073965 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2fe9a5ac-1a3c-43c7-97fa-78dd596e7a69 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.154578 Sep 27 04:28:26.074847 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 414/814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:25 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:26.159969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7842fec9-9ef4-47fb-b9ff-987b5ba3468b", "name": "network-vif-plugged", "status": "completed", "tag": "765ce5cc-318e-4b81-afac-627ffadfa36a"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:26.170249 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:26.170729 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:26.171411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:26.182828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] Creating event network-vif-plugged:765ce5cc-318e-4b81-afac-627ffadfa36a for instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b on hv-cinder-85918 Sep 27 04:28:26.187806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-276b31ea-eaf1-4738-a6aa-2cc0188c50a0 req-e2be3766-bafd-4c9b-92f8-2469d16781bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031498 Sep 27 04:28:26.188599 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 401/815] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:28:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:28:27.096940 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:27.097544 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:27.104454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:27.104938 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:27.105376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:27.106086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:27.106538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:27.106959 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:27.232077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:27.239571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-99eebc44-17d9-4790-8a27-9980ca48fd7a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1367 microversion: 2.1 time: 0.146544 Sep 27 04:28:27.240134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 415/816] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:27 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:28.261396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:28.262287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:28.266559 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:28.267667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:28.276856 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:28.277526 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:28.278115 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:28.278962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:28.279627 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:28.280253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:28.407389 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:28:28.415093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ae5ee033-d725-4443-aba3-a57edd7d4444 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1557 microversion: 2.1 time: 0.155757 Sep 27 04:28:28.415712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 402/817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:28:28 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1557 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:28:28.437619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:28:28.438177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:28:28.439986 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:28:28.445969 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:28.446450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:28.446885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:28.447574 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:28.448028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:28.448449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:28.526527 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-29374e82-73ec-43f0-8550-7bec9c305990 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.092703 Sep 27 04:28:28.527134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 416/818] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:28:28 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:28:29.704439 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7842fec9-9ef4-47fb-b9ff-987b5ba3468b", "tag": "765ce5cc-318e-4b81-afac-627ffadfa36a"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:28:29.714756 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:28:29.715247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:28:29.715711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:28:29.726822 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] Creating event network-changed:765ce5cc-318e-4b81-afac-627ffadfa36a for instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b on hv-cinder-85918 Sep 27 04:28:29.731809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-a40391e0-f3e0-49bd-887f-597d998e67f5 req-ab357bde-f677-40dc-9b73-5dd1f4884960 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030993 Sep 27 04:28:29.732400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 403/819] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:28:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:29:05.920175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:05.920999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=34 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:05.925442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:05.925955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:05.933053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:05.933531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:05.933989 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:05.934673 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:05.935124 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:05.935548 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:05.959675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a3c1a309-bc7c-4eb7-9d29-9e613b4303f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/ips" status: 200 len: 150 microversion: 2.1 time: 0.040848 Sep 27 04:29:05.960215 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 417/820] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:29:05 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/ips => generated 150 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:05.969410 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:05.970177 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=31 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:05.974986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "479a0dc4-bdbb-429d-8219-e3a3e5a11211"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:05.977524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:05.985319 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:05.985826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:05.986265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:05.986976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:05.987428 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:05.987852 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:06.066462 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: POST /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/os-volume_attachments HTTP/1.1 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 74 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:29:06.068206 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {"volumeAttachment": {"volumeId": "479a0dc4-bdbb-429d-8219-e3a3e5a11211"}} Sep 27 04:29:06.069923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'server_id': '7842fec9-9ef4-47fb-b9ff-987b5ba3468b', 'body': {'volumeAttachment': {'volumeId': '479a0dc4-bdbb-429d-8219-e3a3e5a11211'}}} Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:06.071685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:06.073156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 400 to user: Invalid volume: volume 479a0dc4-bdbb-429d-8219-e3a3e5a11211 is already attached to instances: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4c7fde4f-da44-4c82-b309-3535cde87350 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.101748 Sep 27 04:29:06.074602 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 404/821] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:29:05 2022] POST /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:29:06.078285 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:06.078789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:06.085095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:06.085577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:06.086043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:06.086731 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:06.087190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:06.087626 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:06.111909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:29:06.191354 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8c1dcb1e-9c96-4d16-9fae-4371a048e9b6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 204 len: 0 microversion: 2.1 time: 0.116636 Sep 27 04:29:06.192008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 418/822] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:29:06 2022] DELETE /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:29:06.202121 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:06.202782 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:06.209681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:06.210205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:06.210647 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:06.211342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:06.211802 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:06.212232 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:06.342300 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:06.350394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-eb191fb0-e598-45f7-bb3f-778ec8764e2c tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1682 microversion: 2.1 time: 0.153358 Sep 27 04:29:06.350948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 405/823] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:06 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:29:07.374313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:07.375016 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:07.382047 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:07.382534 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:07.382973 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:07.383658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:07.384115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:07.384543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:07.518314 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:07.526515 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5b244ac4-baa5-46d9-b9f5-6c0b6f07ed3b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1682 microversion: 2.1 time: 0.156503 Sep 27 04:29:07.527071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 419/824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:07 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:08.551492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:08.552103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:08.558390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:08.558877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:08.559310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:08.559995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:08.560447 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:08.560878 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:08.662677 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:08.670694 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0aaf5918-21eb-4660-b63f-61cf1d22a01d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 200 len: 1614 microversion: 2.1 time: 0.122970 Sep 27 04:29:08.671270 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 406/825] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:08 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 1614 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:08.880336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7842fec9-9ef4-47fb-b9ff-987b5ba3468b", "name": "network-vif-deleted", "tag": "765ce5cc-318e-4b81-afac-627ffadfa36a"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:08.894671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:08.896043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:08.896543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:08.909693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] Creating event network-vif-deleted:765ce5cc-318e-4b81-afac-627ffadfa36a for instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b on hv-cinder-85918 Sep 27 04:29:08.916484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-b41b8ca7-5461-4a48-a6cc-c981f67c2e28 req-0a74bc9f-14c8-45be-aa96-f4f0b4e1a789 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040264 Sep 27 04:29:08.917300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 420/826] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:09.694091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:09.694710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:09.701529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:09.702048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:09.702487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:09.703229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:09.703684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:09.704108 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:09.718697 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b HTTP/1.1 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:29:09.720247 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:29:09.721813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '7842fec9-9ef4-47fb-b9ff-987b5ba3468b'} Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:29:09.722587 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:29:09.723879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:29:09.725005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:29:09.725005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-29d63463-bc60-4023-beca-6f02af46670f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 404 len: 111 microversion: 2.1 time: 0.032176 Sep 27 04:29:09.725005 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 407/827] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:09 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:29:09.731156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d54beb6b-c12c-43fe-a75c-cdab856aab3a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '.version_select at 0x7fce707d6ee0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:09.751195 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d54beb6b-c12c-43fe-a75c-cdab856aab3a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-368356529" status: 202 len: 0 microversion: 2.1 time: 0.024122 Sep 27 04:29:09.751787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 421/828] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:29:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-368356529 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:29:10.405590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:10.406111 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:10.412458 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:10.412948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:10.413393 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:10.414115 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:10.414604 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:10.415032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:10.494013 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lazy-loading 'services' on Instance uuid 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:29:10.564352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c45c1221-5cac-4f6d-9ee6-5a36eed71779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments/479a0dc4-bdbb-429d-8219-e3a3e5a11211" status: 202 len: 0 microversion: 2.1 time: 0.162461 Sep 27 04:29:10.564926 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 408/829] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:29:10 2022] DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments/479a0dc4-bdbb-429d-8219-e3a3e5a11211 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:29:10.573629 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:10.575917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:10.583225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:10.583708 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:10.584154 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:10.584862 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:10.585325 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:10.585775 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:10.616015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e60dd473-8b8f-4a0d-b7ca-614056a98c0d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046718 Sep 27 04:29:10.616571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 422/830] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:29:10 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:11.629227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:11.631591 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:11.637712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:11.638235 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:11.638672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:11.639357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:11.639814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:11.640237 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:11.671139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-91bb5518-3474-4ac5-a481-6b998ef6cb5b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046559 Sep 27 04:29:11.671743 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 409/831] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:29:11 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:12.684060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:12.686187 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:12.692540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.693024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.693463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:12.694205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.694667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.695095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:12.723023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-37edb7b4-ab3a-46f3-9f57-b2ed37354ffd tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043011 Sep 27 04:29:12.724066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 423/832] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:29:12 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:29:12.785292 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:12.786316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:12.792560 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.793044 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.793755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:12.794441 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.794895 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.795325 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:12.819915 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:29:12.892622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-eaa3aae5-a8fc-4a35-b4b5-d1d56cf59135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 204 len: 0 microversion: 2.1 time: 0.110800 Sep 27 04:29:12.893326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 410/833] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:29:12 2022] DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:29:12.902153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:12.902799 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:12.908724 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.909220 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.909660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:12.910376 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:12.910850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:12.911853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:13.042542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:13.050437 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-90ad2ad7-30a9-4dcb-9c62-df656fd2c8d4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1681 microversion: 2.1 time: 0.152137 Sep 27 04:29:13.050995 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 424/834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:12 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1681 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:14.079424 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:14.080169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:14.088962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.089461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.089940 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.090651 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.091118 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.091549 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.302035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:14.309800 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f7e356ba-2f5c-4cfd-8163-6c4ae656ecc0 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1681 microversion: 2.1 time: 0.239743 Sep 27 04:29:14.310484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 411/835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:14 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1681 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:14.498281 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:14.498836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:14.503693 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:14.506173 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.506658 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.507091 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.507780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.508233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.508656 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.514332 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.514906 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.515433 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.526924 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] Creating event network-changed:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:29:14.531978 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-df2990b8-a4af-472b-8ea7-e0c394583599 req-27f7c86e-4e12-4bc5-aad0-c86ec4e678f1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031863 Sep 27 04:29:14.532890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 412/836] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:14.534450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:29:14.623849 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b0b710fb-ad4a-4c83-83a1-aac997a4ce0b tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 204 len: 0 microversion: 2.1 time: 0.129050 Sep 27 04:29:14.624858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 425/837] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:29:14 2022] DELETE /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:29:14.635399 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:14.636551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:14.642635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.643130 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.643574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.644276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:14.644738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:14.645169 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:14.780755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:14.789017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3e64bb92-2c04-47cf-a464-59e0d39455f3 tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.159250 Sep 27 04:29:14.789577 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 413/838] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:14 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:15.335369 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:15.336165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:15.342442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:15.342937 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:15.343379 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:15.344083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:15.344543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:15.344992 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:15.478729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:15.486668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-61e5d3e1-e906-4abd-ad62-e0c9719cde06 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1681 microversion: 2.1 time: 0.156064 Sep 27 04:29:15.487313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 426/839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:15 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:15.813265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:15.813941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:15.820634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:15.821069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:15.821508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:15.822227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:15.822691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:15.823126 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:16.015583 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:16.023678 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a0ce0504-af49-4674-a416-bdb3a8da803e tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1571 microversion: 2.1 time: 0.214215 Sep 27 04:29:16.025151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 414/840] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:15 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1571 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:16.510467 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:16.511141 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:16.517507 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:16.518028 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:16.518465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:16.519162 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:16.519615 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:16.520261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:16.623449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:16.631181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c6d2db8f-fe24-4a27-ac89-83ef0b63ec78 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1613 microversion: 2.1 time: 0.124478 Sep 27 04:29:16.631756 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 427/841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:16 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1613 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:16.953776 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93c1b78a-1f68-4d8b-a00a-15ed4dfe4779", "name": "network-vif-deleted", "tag": "59621d87-e6fa-445c-98d8-2f933c046f0d"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:16.967158 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:16.968411 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:16.969148 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:16.982488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] Creating event network-vif-deleted:59621d87-e6fa-445c-98d8-2f933c046f0d for instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 on hv-cinder-85918 Sep 27 04:29:16.988290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-517ee62d-fc62-4d55-93a0-9a7559443d4c req-9b9fbde0-f9dd-47ec-9fe3-5f82f7ce6781 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038057 Sep 27 04:29:16.989484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 415/842] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:17.052496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:17.053104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:17.059505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:17.060082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:17.060637 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:17.061591 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:17.062227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:17.062783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:17.163455 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ebc33b6c372434287b632d70f0a9e83', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e595829629a4dd4a7fccbdabc896cb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:17.173378 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-85edf74c-48c4-4752-8d89-7aecfceac1aa tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 200 len: 1384 microversion: 2.1 time: 0.124635 Sep 27 04:29:17.173988 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 428/843] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:17 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 1384 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:17.656236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:17.657800 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:17.666384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:17.667221 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:17.667891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:17.669101 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:17.670028 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:17.670716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:17.707885 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:17.708523 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:17.709032 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:17.709817 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lazy-loading 'fault' on Instance uuid 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:29:17.762652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:17.770104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bd569ae7-da44-45f4-82aa-544b5a33ec95 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 200 len: 1331 microversion: 2.1 time: 0.119064 Sep 27 04:29:17.770698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 416/844] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:17 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 1331 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:18.201560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:18.202450 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] [instance: 3063ca42-9212-4df2-af49-0d7d4a88c0ee] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:18.209304 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:18.209812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:18.210252 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:18.210943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:18.211392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:18.211814 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:18.226108 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] HTTP exception thrown: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Request method failure captured: Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee HTTP/1.1 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnsK-rR1U20xcZqV0hFrR4jW6HfUlVUzc5u5wnRGFCQCUk4eNrQc3dRfQaCDyNc6YVuh-NwVKNLm5C0nnY77lOdDr9glcbY8IntAeChrgaDQCUMCOu4h0GoP7ZBgaX2n4RpzjvvVe5P8J7Y6-KzT8hnwsm47DgkDY75zniZ6GGtJ4js Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 1e595829629a4dd4a7fccbdabc896cb0 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-TestShelveInstance-1290579173 Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:29:18.227784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1e595829629a4dd4a7fccbdabc896cb0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e595829629a4dd4a7fccbdabc896cb0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1e595829629a4dd4a7fccbdabc896cb0", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e595829629a4dd4a7fccbdabc896cb0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-TestShelveInstance-1290579173 Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 1e595829629a4dd4a7fccbdabc896cb0 Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-TestShelveInstance-1290579173 Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-TestShelveInstance-1290579173-project Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 5ebc33b6c372434287b632d70f0a9e83 Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-TestShelveInstance-1290579173-project Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. Sep 27 04:29:18.229458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '3063ca42-9212-4df2-af49-0d7d4a88c0ee'} Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:29:18.230601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. Sep 27 04:29:18.232192 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.233886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Returning 404 to user: Instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:29:18.233886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0a81a240-6821-4a8d-af0c-f5e9682d2f2d tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee" status: 404 len: 111 microversion: 2.1 time: 0.036586 Sep 27 04:29:18.233886 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 429/845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:18 2022] GET /compute/v2.1/servers/3063ca42-9212-4df2-af49-0d7d4a88c0ee => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:29:18.406326 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4bf47942-0ef7-44ed-ac81-6ea85159311a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] Calling method '.version_select at 0x7fce7091a550>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:18.429390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4bf47942-0ef7-44ed-ac81-6ea85159311a tempest-TestShelveInstance-1290579173 tempest-TestShelveInstance-1290579173-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1512923076" status: 202 len: 0 microversion: 2.1 time: 0.026912 Sep 27 04:29:18.433458 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 417/846] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 04:29:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1512923076 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:29:18.795138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:18.795865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:18.801822 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:18.802320 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:18.802762 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:18.803454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:18.803912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:18.804341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:18.818432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 HTTP/1.1 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:29:18.819972 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:29:18.821418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '93c1b78a-1f68-4d8b-a00a-15ed4dfe4779'} Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:29:18.822250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:29:18.823486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:29:18.825293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:29:18.825293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5e284459-ca2c-403b-9864-edfa68cbfb1a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 404 len: 111 microversion: 2.1 time: 0.030524 Sep 27 04:29:18.825293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 430/847] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:18 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:29:18.830331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1463b212-5f92-4bcf-beb8-227408bc7aa3 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '.version_select at 0x7fce70985940>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:18.849644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1463b212-5f92-4bcf-beb8-227408bc7aa3 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-516491899" status: 202 len: 0 microversion: 2.1 time: 0.023106 Sep 27 04:29:18.850297 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 418/848] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:29:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-516491899 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:29:19.005895 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3063ca42-9212-4df2-af49-0d7d4a88c0ee", "name": "network-vif-deleted", "tag": "eb918bb7-1383-468a-8553-7476aae953bb"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:19.015468 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:19.016015 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:19.016393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:19.032974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] Creating event network-vif-deleted:eb918bb7-1383-468a-8553-7476aae953bb for instance 3063ca42-9212-4df2-af49-0d7d4a88c0ee on hv-cinder-85918 Sep 27 04:29:19.038667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-4e75edcd-cbc6-415f-b81b-522c721fed98 req-730a4842-0a33-477b-83ba-197af5b15875 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036639 Sep 27 04:29:19.039261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 431/849] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:19.802718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5dde9833-6509-4898-b723-b772e94878b2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: .version_select at 0x7fce70952e50>, body: {"keypair": {"name": "tempest-keypair-47216472", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB5ohUnffplKIFm245pMfTadxOVpvXBY8CsoMJPKF06N7TY+5Whvl3OWLjrTP7TbSdF19Fr5gN0iCzu3D/3h08ophPNJA9D2yicaRD4jC9bcpTkacnRv+ATKUIk/I/omCw=="}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:19.822152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-5dde9833-6509-4898-b723-b772e94878b2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:29:19.826467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-5dde9833-6509-4898-b723-b772e94878b2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:29:19.854077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5dde9833-6509-4898-b723-b772e94878b2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.055701 Sep 27 04:29:19.854768 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 419/850] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:29:19 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:22.012363 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-167943333", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cba0fe06-d029-4450-9dfd-fb0123caa50b"}], "security_groups": [{"name": "tempest-securitygroup--1985755080"}], "key_name": "tempest-keypair-47216472", "user_data": "}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:22.177439 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] validate_networks() for [('cba0fe06-d029-4450-9dfd-fb0123caa50b', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:29:22.813085 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:29:22.813682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:29:22.814311 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:29:22.814836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:29:22.815263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:29:22.824396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:22.824880 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:22.825313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:22.826073 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:22.826582 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:22.826978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:22.845498 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:29:22.850066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:29:22.861603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:29:22.864272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:29:22.915716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-957a7dd0-460f-4093-811f-abd3c81a2783 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.908759 Sep 27 04:29:22.923359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 432/851] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:29:22 2022] POST /compute/v2.1/servers => generated 402 bytes in 917 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:29:22.929176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3d667a08-6593-4da0-af2d-127f7d95ab50 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:22.929827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3d667a08-6593-4da0-af2d-127f7d95ab50 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:23.006703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3d667a08-6593-4da0-af2d-127f7d95ab50 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:23.016925 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3d667a08-6593-4da0-af2d-127f7d95ab50 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1239 microversion: 2.1 time: 0.092299 Sep 27 04:29:23.017500 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 420/852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:22 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1239 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:24.042138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:24.042828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:24.050256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:24.051094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:24.051603 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:24.052299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:24.053360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:24.054123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:24.163422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:24.172644 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-58a5d7c0-cc52-4fe9-8118-b688b1c55b12 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1309 microversion: 2.1 time: 0.137269 Sep 27 04:29:24.173340 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 433/853] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:24 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1309 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:25.196601 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:25.197412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:25.217717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:25.218243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:25.219079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:25.220434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:25.220900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:25.221333 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:25.501263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:25.511039 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f0071db7-7abf-4469-9a74-516322eb98c4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.318191 Sep 27 04:29:25.528816 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 421/854] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:25 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:26.072622 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af03ef64-1418-4b7a-a0ee-1294cedb21a1", "tag": "9cf35c89-da17-4b43-a211-771d19ddbbf2"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:26.097336 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:26.100659 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:26.105098 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:26.133681 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] Creating event network-changed:9cf35c89-da17-4b43-a211-771d19ddbbf2 for instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 on hv-cinder-85918 Sep 27 04:29:26.152410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-1c425b34-9662-4534-84f4-ad7bcd22aded req-a5de78f0-7bd9-4508-9892-f4a4a3bfee53 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083414 Sep 27 04:29:26.154779 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 434/855] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:26.536970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:26.538185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:26.545089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:26.545949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:26.546390 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:26.547088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:26.547545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:26.547970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:26.764969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:26.774536 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-11a185f1-e4e8-478a-a923-e478da61329a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.241367 Sep 27 04:29:26.775145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 422/856] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:26 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:27.799221 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:27.799932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:27.822633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:27.823382 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:27.826169 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:27.827273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:27.827914 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:27.828485 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:27.863307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:28.012235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:28.019790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cd4a4b47-1a18-4ab7-82da-03512ff684fa tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.225173 Sep 27 04:29:28.020382 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 435/857] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:27 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:29.045155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:29.045877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:29.053738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:29.054336 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:29.054851 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:29.055796 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:29.056454 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:29.056886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:29.195254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:29.199549 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:29.206157 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bf163281-590d-4b54-bc24-43230cac5b59 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.167504 Sep 27 04:29:29.206734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 423/858] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:29 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:30.231150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:30.232414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:30.240283 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:30.241224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:30.241676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:30.242419 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:30.242883 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:30.243308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:30.490367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:30.502699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-459e1510-026b-4472-8d3c-49e6a2361c88 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.275357 Sep 27 04:29:30.505919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 436/859] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:30 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:29:30.506029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af03ef64-1418-4b7a-a0ee-1294cedb21a1", "name": "network-vif-plugged", "status": "completed", "tag": "9cf35c89-da17-4b43-a211-771d19ddbbf2"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:30.528617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:30.529566 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:30.530196 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:30.543113 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] Creating event network-vif-plugged:9cf35c89-da17-4b43-a211-771d19ddbbf2 for instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 on hv-cinder-85918 Sep 27 04:29:30.549076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-c74b2fd4-6797-4e58-832f-05c072d9cc4f req-a0e4c4ab-cceb-4a3d-b2da-be9008559158 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060541 Sep 27 04:29:30.550272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 424/860] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:29:31.529857 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:31.530752 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:31.538229 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:31.538721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:31.539484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:31.540182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:31.540796 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:31.542004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:31.773540 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:31.784981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-bf449f96-93a3-4d4f-8087-ebf7798887dc tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1365 microversion: 2.1 time: 0.258814 Sep 27 04:29:31.800815 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 437/861] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:31 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1365 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:32.809823 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:32.811076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:32.819150 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:32.819873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:32.820318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:32.821023 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:32.821835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:32.822524 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:32.971755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:32.981069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d35a1603-9059-49d6-ba01-4e6328699a86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1554 microversion: 2.1 time: 0.175273 Sep 27 04:29:32.981855 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 425/862] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:32 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1554 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:33.005793 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:33.007892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:33.010291 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:33.011580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:33.018213 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:33.018705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:33.019322 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:33.020034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:33.021197 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:33.021630 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:33.105489 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-26e16354-b873-421a-abc9-af6842a1c078 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.105438 Sep 27 04:29:33.106121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 438/863] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:29:32 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:34.376845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af03ef64-1418-4b7a-a0ee-1294cedb21a1", "tag": "9cf35c89-da17-4b43-a211-771d19ddbbf2"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:34.386645 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:34.390417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:34.391153 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:34.391777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:34.413525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] Creating event network-changed:9cf35c89-da17-4b43-a211-771d19ddbbf2 for instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 on hv-cinder-85918 Sep 27 04:29:34.420828 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-4cf5701b-7579-4c7a-bee1-1fd535b792bf req-2538d81b-f2cc-48cf-8a94-8a757bc22903 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047501 Sep 27 04:29:34.421492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 426/864] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:38.672983 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-847620db-6108-4180-8f94-648ae5b790f5 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Action: 'create', calling method: .version_select at 0x7fce70a63c10>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1258945985", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEhvphIGrGFlpiasvBpgbEXWIcxVBYc9a6MCFT2/apj63h7LFUe+3wmnFtVUm9nZOricQMT1JDllu8RYTGhp9H2ApAcYMQf+DfPgh2Z4SJVOxGyvzqpOhLyiST3YwCdaEg=="}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:38.678783 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:38.691264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-847620db-6108-4180-8f94-648ae5b790f5 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:29:38.696388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-847620db-6108-4180-8f94-648ae5b790f5 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for user 44a710d801334b20b9f433eaad61cbc6 and project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'key_pairs'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:29:38.720129 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-847620db-6108-4180-8f94-648ae5b790f5 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.245295 Sep 27 04:29:38.727130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 439/865] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:29:38 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:55.076240 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1,nova_api:DELETE=2,nova_api:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:55.076994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=10 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:55.081165 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-460153178", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1258945985", "security_groups": [{"name": "tempest-secgroup-smoke-1621255540"}], "block_device_mapping_v2": [{"uuid": "c535295c-02ac-420a-8d45-3ae388060e4f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "df4f6840-004b-4387-a687-683e2895186f"}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:55.204287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] validate_networks() for [('df4f6840-004b-4387-a687-683e2895186f', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:29:55.564315 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:29:55.564744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:29:55.565238 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:29:55.565635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:29:55.566078 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:29:55.574871 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:55.575358 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:55.575798 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:55.576533 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:55.577002 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:55.577456 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:55.594224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:29:55.598128 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for user 44a710d801334b20b9f433eaad61cbc6 and project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:29:55.656284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:29:55.658549 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c535295c-02ac-420a-8d45-3ae388060e4f',volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:29:55.785544 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2c3793cf-bf0e-45ad-bca1-36056f7effde tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.710755 Sep 27 04:29:55.786792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 427/866] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:29:55 2022] POST /compute/v2.1/servers => generated 406 bytes in 713 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:29:55.795686 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=18 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:55.796859 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=4 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:55.797634 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=10,nova_api:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:29:55.801524 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-24b247ea-fd12-4d40-bd9f-f797b12b237c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:55.802151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-24b247ea-fd12-4d40-bd9f-f797b12b237c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:55.857145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-24b247ea-fd12-4d40-bd9f-f797b12b237c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:55.864066 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-24b247ea-fd12-4d40-bd9f-f797b12b237c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1094 microversion: 2.1 time: 0.069916 Sep 27 04:29:55.864631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 440/867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:55 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1094 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:56.886361 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:56.886985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:56.893923 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:56.894428 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:56.894873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:56.895576 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:56.896225 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:56.896658 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:56.960129 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:56.969703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-abee0305-1bf8-4a52-9c22-db5149db64fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1210 microversion: 2.1 time: 0.087392 Sep 27 04:29:56.970494 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 428/868] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:56 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1210 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:57.993622 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:57.994434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:58.001026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:58.001547 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:58.002011 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:58.002712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:58.003172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:58.003599 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:58.199179 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:58.206852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e0bf8ce4-fbd2-449b-9d37-819dfb41402b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.217299 Sep 27 04:29:58.207760 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 441/869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:57 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:29:58.251698 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "298d0196-0c87-4bfd-a862-be6cbe3a28f5", "tag": "1712560b-01e8-4b96-8c66-8877d22eba22"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:29:58.262928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:58.263413 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:58.263847 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:58.278883 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] Creating event network-changed:1712560b-01e8-4b96-8c66-8877d22eba22 for instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 on hv-cinder-85918 Sep 27 04:29:58.283849 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-a48f0ddc-e123-436e-b440-b69a925ece11 req-7e49b065-2b46-4170-9dca-a028a5ea3ce4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035980 Sep 27 04:29:58.284811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 429/870] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:29:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:29:59.230566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:29:59.231177 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:29:59.237556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:59.238068 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:59.238487 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:59.239207 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:29:59.239662 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:29:59.240086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:29:59.545329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:29:59.553443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ca052142-f01a-41a9-ae37-35d0d4138ff4 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.326992 Sep 27 04:29:59.554053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 442/871] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:29:59 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:00.577426 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:00.578154 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:00.584272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:00.584761 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:00.585198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:00.585909 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:00.586371 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:00.586803 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:00.720447 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:00.728606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8a640e7f-0be6-40ff-9713-9b989ef7e2ad tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.155882 Sep 27 04:30:00.729171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 430/872] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:00 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:01.751682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:01.752292 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:01.759135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:01.759617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:01.760053 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:01.760736 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:01.761190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:01.761611 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:01.898184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:01.905763 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-94bfe44b-8086-47de-97d0-b8d791322e48 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.157985 Sep 27 04:30:01.906331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 443/873] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:01 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:02.929009 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:02.929647 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:02.936048 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:02.936540 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:02.936982 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:02.937671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:02.938155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:02.938584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:03.117053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:03.124692 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-04549f06-0ffb-4097-b6e2-1615b61d0bc8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.200172 Sep 27 04:30:03.125252 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 431/874] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:02 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:04.147351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:04.147929 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:04.154403 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:04.154887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:04.155323 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:04.156009 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:04.156464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:04.156889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:04.286789 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:04.294427 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6ee1918d-a524-465a-9f52-2784ef7774af tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.151023 Sep 27 04:30:04.295255 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 444/875] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:04 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:05.318282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:05.318905 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:05.325383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:05.325911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:05.326394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:05.327088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:05.327554 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:05.327986 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:05.456837 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:05.464476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-14d4edf7-8c8a-4de1-a8f1-c81f6b28a7a3 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.150588 Sep 27 04:30:05.465035 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 432/876] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:05 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:06.487393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:06.488004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:06.490432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:06.495633 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:06.496121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:06.496560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:06.497247 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:06.497706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:06.498156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:06.637966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:06.645561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b9cbb4c7-d22e-44d3-a881-67c8216d862b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.162009 Sep 27 04:30:06.646144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 445/877] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:06 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:07.667968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:07.668606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:07.670748 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:07.671581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:07.677873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:07.678363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:07.678806 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:07.679486 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:07.679941 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:07.680363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:07.811418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:07.819537 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fc7e36eb-48b9-4730-aae4-659c973a9aea tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.156216 Sep 27 04:30:07.820098 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 433/878] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:07 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:08.644725 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "298d0196-0c87-4bfd-a862-be6cbe3a28f5", "name": "network-vif-plugged", "status": "completed", "tag": "1712560b-01e8-4b96-8c66-8877d22eba22"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:08.655545 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:08.656061 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:08.656497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:08.668891 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] Creating event network-vif-plugged:1712560b-01e8-4b96-8c66-8877d22eba22 for instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 on hv-cinder-85918 Sep 27 04:30:08.674150 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-99b02ffe-1f63-4dc3-8e04-a797f6581232 req-a8646e00-4c82-4b5a-8e1e-b45a18856129 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033138 Sep 27 04:30:08.674997 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 446/879] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:08.842470 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:08.843085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:08.849384 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:08.849902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:08.850339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:08.851030 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:08.851488 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:08.851912 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:08.980955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:08.988562 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a1754a9a-4b64-41ab-be90-774be7c944c0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.150929 Sep 27 04:30:08.989122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 434/880] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:08 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:30:10.012235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:10.012958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:10.019121 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:10.019602 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:10.020041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:10.021238 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:10.021696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:10.022153 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:10.154865 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:10.163086 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-b9275793-258d-454d-843c-13204cd16c9a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1266 microversion: 2.1 time: 0.155526 Sep 27 04:30:10.163710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 447/881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:10 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:11.185955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:11.186482 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:11.193464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:11.193987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:11.194431 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:11.195123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:11.195584 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:11.196016 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:11.327624 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:11.335910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-a8132abb-74fb-4c8d-8aa1-e912f2eb765b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1452 microversion: 2.1 time: 0.153915 Sep 27 04:30:11.336477 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 435/882] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:11 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1452 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:11.357941 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:11.358553 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:11.365502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:11.366032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:11.366505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:11.367227 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:11.367690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:11.368120 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:11.495906 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:11.503596 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-863f8b91-d6c7-490b-bdb5-e654359449d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1452 microversion: 2.1 time: 0.149513 Sep 27 04:30:11.504166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 448/883] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:11 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1452 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:13.099747 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "298d0196-0c87-4bfd-a862-be6cbe3a28f5", "tag": "1712560b-01e8-4b96-8c66-8877d22eba22"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:13.109331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:13.109853 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:13.110295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:13.121578 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] Creating event network-changed:1712560b-01e8-4b96-8c66-8877d22eba22 for instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 on hv-cinder-85918 Sep 27 04:30:13.126659 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-91539a82-8744-42de-b112-a0e091dc4ac3 req-a5619387-8f2a-432c-b96a-06ebbb24b4f0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030591 Sep 27 04:30:13.127249 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 436/884] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:30:16.393344 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:16.394018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:16.401136 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:16.401617 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:16.402075 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:16.402759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:16.403211 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:16.403632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:16.427184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a88d6048-4847-41b0-aa14-d8eb619e8b35 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/ips" status: 200 len: 149 microversion: 2.1 time: 0.037739 Sep 27 04:30:16.427741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 449/885] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:30:16 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/ips => generated 149 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:17.756380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "eede965d-7d0d-4896-bb8a-ab3f105fd486"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:17.758794 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:17.764778 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:17.765256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:17.765692 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:17.766391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:17.766846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:17.767269 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:18.006732 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ec6fc9c0-5655-4061-b7fb-bb8dcf0bb5ae tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.255572 Sep 27 04:30:18.012747 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 437/886] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:30:17 2022] POST /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments => generated 194 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:24.492632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "eede965d-7d0d-4896-bb8a-ab3f105fd486"}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:24.495006 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:24.500946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.502435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.502850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.503544 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.504134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.504560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.582798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: POST /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments HTTP/1.1 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 74 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:30:24.584737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {"volumeAttachment": {"volumeId": "eede965d-7d0d-4896-bb8a-ab3f105fd486"}} Sep 27 04:30:24.586477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'server_id': 'af03ef64-1418-4b7a-a0ee-1294cedb21a1', 'body': {'volumeAttachment': {'volumeId': 'eede965d-7d0d-4896-bb8a-ab3f105fd486'}}} Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:30:24.587416 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:24.588926 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 400 to user: Invalid volume: volume eede965d-7d0d-4896-bb8a-ab3f105fd486 is already attached to instances: af03ef64-1418-4b7a-a0ee-1294cedb21a1 {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e09bfbfd-74f9-458a-a339-0984b22cd972 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.097701 Sep 27 04:30:24.590518 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 450/887] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:30:24 2022] POST /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 04:30:24.654993 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:24.655575 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:24.662704 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.663192 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.663631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.664316 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.664768 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.665189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.746216 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lazy-loading 'services' on Instance uuid af03ef64-1418-4b7a-a0ee-1294cedb21a1 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:30:24.816236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f35e28dd-ad2f-4163-9345-cf05ba383087 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments/eede965d-7d0d-4896-bb8a-ab3f105fd486" status: 202 len: 0 microversion: 2.1 time: 0.165891 Sep 27 04:30:24.816839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 438/888] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:30:24 2022] DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments/eede965d-7d0d-4896-bb8a-ab3f105fd486 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:30:24.826560 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:24.829077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:24.836852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.837418 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.837952 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.838915 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:24.839370 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:24.839796 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:24.872401 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2ef4a7dd-bf99-4584-8695-a6f29cac6c86 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051021 Sep 27 04:30:24.872956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 451/889] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:30:24 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:25.886717 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:25.889076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:25.891056 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:25.897295 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:25.897827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:25.898273 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:25.898970 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:25.899428 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:25.899882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:25.902827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:25.932653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-4c34aeb8-448b-440c-809d-a827fdb532d6 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050851 Sep 27 04:30:25.933195 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 439/890] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:30:25 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:30:26.000495 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:26.001032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:26.003422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=27 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:26.009431 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:26.009956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:26.010399 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:26.011123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:26.011584 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:26.012018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:26.016204 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=61 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:26.039852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:30:26.111956 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8b62244d-bfc7-4958-bc07-403a9c479f99 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 204 len: 0 microversion: 2.1 time: 0.114922 Sep 27 04:30:26.112554 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 452/891] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:30:25 2022] DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:30:26.122404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:26.123266 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:26.129230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:26.129718 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:26.130192 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:26.130880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:26.131335 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:26.131758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:26.261310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:26.269406 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-146e385e-73e4-478a-9dde-546f1f65fc37 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1679 microversion: 2.1 time: 0.152378 Sep 27 04:30:26.269972 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 440/892] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:26 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:27.293943 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:27.294675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:27.301115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:27.301595 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:27.302064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:27.302749 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:27.303202 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:27.303627 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:27.429985 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:27.437674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-28508753-7826-4a77-9647-5c74a94d3b80 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1679 microversion: 2.1 time: 0.148257 Sep 27 04:30:27.438263 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 453/893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:27 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1679 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:28.461710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:28.462487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:28.469540 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:28.469997 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:28.471052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:28.471763 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:28.472253 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:28.472683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:28.537436 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:28.547244 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-46e1ed8c-8267-4e3d-887e-58892816b052 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 200 len: 1611 microversion: 2.1 time: 0.089134 Sep 27 04:30:28.554509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 441/894] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:28 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 1611 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:28.902329 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af03ef64-1418-4b7a-a0ee-1294cedb21a1", "name": "network-vif-deleted", "tag": "9cf35c89-da17-4b43-a211-771d19ddbbf2"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:28.912884 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:28.913453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:28.914194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:28.926004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] Creating event network-vif-deleted:9cf35c89-da17-4b43-a211-771d19ddbbf2 for instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 on hv-cinder-85918 Sep 27 04:30:28.930840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-0cdd21a0-4526-4cf7-abb5-dbf86fed5441 req-40fa1a61-5752-4fc1-8aee-a7791e5e6007 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032527 Sep 27 04:30:28.931506 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 454/895] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:29.570984 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:29.571527 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:29.578087 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:29.578589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:29.579055 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:29.579750 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:29.580210 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:29.580646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:29.601499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 HTTP/1.1 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:30:29.603213 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:30:29.604765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'af03ef64-1418-4b7a-a0ee-1294cedb21a1'} Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:30:29.605534 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:30:29.607943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:30:29.609691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:30:29.609691 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-5eb3a16d-a5f1-449c-bfb8-fe4d4e8e5e1d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 404 len: 111 microversion: 2.1 time: 0.037966 Sep 27 04:30:29.612432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 442/896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:29 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:30:29.615200 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-aa4fa4e8-f862-48f8-a800-f942e41bb779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '.version_select at 0x7fce709021f0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:29.634112 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-aa4fa4e8-f862-48f8-a800-f942e41bb779 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-47216472" status: 202 len: 0 microversion: 2.1 time: 0.023873 Sep 27 04:30:29.634666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 455/897] 10.222.0.26 () {58 vars in 1215 bytes} [Tue Sep 27 04:30:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-47216472 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:30:30.265464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d846c99d-5495-48bd-bc35-748662318b30 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: .version_select at 0x7fce7084d280>, body: {"keypair": {"name": "tempest-keypair-687461414", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL9o9OffXVdrNrCrfSML8C8oJW3W4WFjKIaL4RNwfBCK7tKcf2+VUTNk4BJUGCn14EIVw9G+MjTX0sK6FHwKQrDtA0AlN+4pj6jjdeezYa9YvJu9sJSR+lZ7USUIkdWeOA=="}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:30.281299 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-d846c99d-5495-48bd-bc35-748662318b30 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:30:30.285053 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-d846c99d-5495-48bd-bc35-748662318b30 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'key_pairs'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:30:30.306800 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d846c99d-5495-48bd-bc35-748662318b30 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045294 Sep 27 04:30:30.307383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 443/898] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 04:30:30 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:31.545023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-794985356", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cba0fe06-d029-4450-9dfd-fb0123caa50b"}], "security_groups": [{"name": "tempest-securitygroup--1007767043"}], "key_name": "tempest-keypair-687461414", "user_data": "}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:31.646360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] validate_networks() for [('cba0fe06-d029-4450-9dfd-fb0123caa50b', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:30:32.015214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:30:32.015660 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:30:32.016245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:30:32.016653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:30:32.017059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:30:32.025671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:32.026180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:32.026607 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:32.027319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:32.027770 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:32.028198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:32.044508 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:30:32.048858 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Getting quotas for user dfb35bfba72546f3853b9dd9ef7f5078 and project d7c74ffec54740eca3cd6b707f1cb732. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:30:32.059449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:30:32.061819 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:30:32.106739 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f576cae2-59b9-4278-97d9-582a59737bf4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.566228 Sep 27 04:30:32.107488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 456/899] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:30:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 568 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:30:32.119302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-d27ff8ae-99ff-4ab9-a1db-cc549c4d34db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:32.119917 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-d27ff8ae-99ff-4ab9-a1db-cc549c4d34db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:32.176302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-d27ff8ae-99ff-4ab9-a1db-cc549c4d34db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:32.182977 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-d27ff8ae-99ff-4ab9-a1db-cc549c4d34db tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1240 microversion: 2.1 time: 0.067949 Sep 27 04:30:32.183538 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 444/900] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:32 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1240 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:33.207424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:33.208043 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:33.215156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:33.215748 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:33.216254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:33.217055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:33.217577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:33.218104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:33.293961 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:33.302564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a4babb33-0f94-4f05-9bc7-8234191ef844 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1310 microversion: 2.1 time: 0.100587 Sep 27 04:30:33.303133 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 457/901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:33 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1310 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:34.325801 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:34.326418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:34.332861 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:34.333350 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:34.333807 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:34.334499 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:34.334951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:34.335373 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:34.466330 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:34.474089 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b07abf37-a0db-4dc9-b460-3e520f67d5d7 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.152198 Sep 27 04:30:34.474661 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 445/902] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:34 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:34.613347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e293adc2-d0ac-46d7-92a9-a61e55ca2f5e", "tag": "ba58590d-7cc5-40df-a30e-19ed0c34f9b2"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:34.630456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:34.630932 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:34.631359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:34.644491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] Creating event network-changed:ba58590d-7cc5-40df-a30e-19ed0c34f9b2 for instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e on hv-cinder-85918 Sep 27 04:30:34.649767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-43f4a833-d5b3-4778-83a3-6360ecb2f06b req-c2fa9f1f-523f-45f7-bc2d-1e6b98d89bf8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042878 Sep 27 04:30:34.650581 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 458/903] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:35.497051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:35.497668 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:35.504605 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:35.505090 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:35.505529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:35.506245 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:35.506705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:35.507134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:35.636928 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:35.645117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0e4243cb-82a3-4bf4-bc53-5f440c3fc448 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.151935 Sep 27 04:30:35.645673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 446/904] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:35 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:36.668835 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:36.669486 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:36.676007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:36.676491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:36.676971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:36.677612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:36.678092 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:36.678514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:36.810308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:36.818178 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ed3addcb-6966-4c45-9005-65411ae2524b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.153861 Sep 27 04:30:36.818790 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 459/905] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:36 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:30:37.840710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:37.841463 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:37.847725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:37.848205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:37.848635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:37.849319 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:37.849790 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:37.850223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:37.978388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:37.986398 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7eb4cf4b-c0e1-45a5-8401-9225eeea106b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.149564 Sep 27 04:30:37.986955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 447/906] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:37 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:38.745720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e293adc2-d0ac-46d7-92a9-a61e55ca2f5e", "name": "network-vif-plugged", "status": "completed", "tag": "ba58590d-7cc5-40df-a30e-19ed0c34f9b2"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:38.756214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:38.756696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:38.757130 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:38.771404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] Creating event network-vif-plugged:ba58590d-7cc5-40df-a30e-19ed0c34f9b2 for instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e on hv-cinder-85918 Sep 27 04:30:38.776514 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-04e6ee7d-40b8-4d17-a137-c633e0599a82 req-0e627844-1da4-4583-8608-90ecaef9176a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034487 Sep 27 04:30:38.777102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 460/907] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:39.009442 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:39.010103 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:39.017163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:39.017656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:39.018125 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:39.018823 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:39.019283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:39.019712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:39.148918 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:39.156513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-57164048-dda2-4819-b962-eae466a2a54a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.151002 Sep 27 04:30:39.157071 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 448/908] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:39 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:40.179453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:40.180064 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:40.186105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:40.186586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:40.187021 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:40.187696 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:40.188151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:40.188569 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:40.317877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:40.325655 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a82afb01-6060-4c9b-9db7-01331eade0a1 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1366 microversion: 2.1 time: 0.150962 Sep 27 04:30:40.326283 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 461/909] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:40 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:41.348758 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:41.349379 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:41.356057 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:41.356552 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:41.356992 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:41.357695 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:41.358194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:41.358622 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:41.493271 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:30:41.501568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ab84f647-fcd0-4b3e-84f5-66685d727c52 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1556 microversion: 2.1 time: 0.156715 Sep 27 04:30:41.502156 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 449/910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:30:41 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1556 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:30:41.524134 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:30:41.524642 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:30:41.532247 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:41.532729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:41.533332 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:41.534234 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:41.534843 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:41.535425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:41.612729 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cd55bd13-6644-451a-a665-8951926105a2 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.092346 Sep 27 04:30:41.613319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 462/911] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 04:30:41 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:30:42.828921 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e293adc2-d0ac-46d7-92a9-a61e55ca2f5e", "tag": "ba58590d-7cc5-40df-a30e-19ed0c34f9b2"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:30:42.836693 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:30:42.844094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:30:42.844586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:30:42.845019 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:30:42.857703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] Creating event network-changed:ba58590d-7cc5-40df-a30e-19ed0c34f9b2 for instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e on hv-cinder-85918 Sep 27 04:30:42.865106 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-88109c2d-ff9a-423e-95a8-ee7017bd1a63 req-16861bcd-db11-4897-b682-dde86c863ede service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040092 Sep 27 04:30:42.866386 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 450/912] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:30:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:18.981276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:18.982156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:18.983966 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:18.984836 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=49,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:18.989024 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:18.989516 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:18.998933 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:18.999525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:18.999968 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:19.000653 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:19.001112 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:19.001538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:19.027315 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-993ee0c4-1bfe-4fb7-b3c6-e90232e9a5f4 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/ips" status: 200 len: 149 microversion: 2.1 time: 0.046840 Sep 27 04:31:19.027869 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 463/913] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:31:18 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/ips => generated 149 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:20.358835 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:20.359606 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=51 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:31:20.364239 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "60b5b023-e379-451e-a5b1-0a6cb2cf2d99"}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:31:20.367274 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:20.374585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:20.375074 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:20.375508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:20.376194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:20.376649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:20.377073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:20.621710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-0b90db9f-1a63-41b9-8d13-d204c96b0823 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "POST /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.264913 Sep 27 04:31:20.624559 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 451/914] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 04:31:20 2022] POST /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments => generated 194 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:28.257039 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:28.257546 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:28.263866 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:28.264346 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:28.264780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:28.265464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:28.265949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:28.266375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:28.344331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lazy-loading 'services' on Instance uuid e293adc2-d0ac-46d7-92a9-a61e55ca2f5e {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:31:28.414537 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-52cf7d23-fcc1-46ea-bb77-74dfaf77778b tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments/60b5b023-e379-451e-a5b1-0a6cb2cf2d99" status: 202 len: 0 microversion: 2.1 time: 0.161223 Sep 27 04:31:28.415105 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 464/915] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 04:31:28 2022] DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments/60b5b023-e379-451e-a5b1-0a6cb2cf2d99 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:31:28.424548 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:28.426967 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:28.433896 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:28.434394 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:28.434840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:28.435688 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:28.436001 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:28.436434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:28.466529 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3f7c51ae-02df-4b8d-aab7-5e9fb9146135 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046792 Sep 27 04:31:28.467085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 452/916] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:31:28 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:29.480842 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:29.483231 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:29.489351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.489866 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.490300 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.490975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.491424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.491840 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.519502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0437fecd-fc10-43be-ad77-3405519bb78d tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043679 Sep 27 04:31:29.520040 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 465/917] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 04:31:29 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 04:31:29.582448 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:29.582968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:29.589809 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.590303 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.590746 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.591432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.591894 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.592321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.617528 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:31:29.692388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-30546423-3551-4c56-a310-4e4605f6af68 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 204 len: 0 microversion: 2.1 time: 0.113320 Sep 27 04:31:29.693438 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 453/918] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:31:29 2022] DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:31:29.702194 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:29.702924 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:29.709404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.709919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.710360 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.711041 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:29.711496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:29.711919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:29.841398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:31:29.849001 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-737463db-c7b4-41df-af26-b055e439eafb tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1680 microversion: 2.1 time: 0.150766 Sep 27 04:31:29.849586 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 466/919] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:29 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:31:30.873091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:30.873816 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:30.880313 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:30.880799 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:30.881236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:30.881954 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:30.882412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:30.882840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:31.013214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:31:31.021804 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8a92eb26-05cc-407b-aa0d-6d8165147342 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1680 microversion: 2.1 time: 0.152987 Sep 27 04:31:31.022359 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 454/920] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:30 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:31:32.045850 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:32.046461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:32.053613 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:32.054147 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:32.054587 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:32.055276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:32.055731 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:32.056148 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:32.126214 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfb35bfba72546f3853b9dd9ef7f5078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd7c74ffec54740eca3cd6b707f1cb732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:31:32.133837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-04e75823-db3f-49f8-8f5e-11d5b9177428 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 200 len: 1612 microversion: 2.1 time: 0.091837 Sep 27 04:31:32.134406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 467/921] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:32 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 1612 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:31:32.367489 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e293adc2-d0ac-46d7-92a9-a61e55ca2f5e", "name": "network-vif-deleted", "tag": "ba58590d-7cc5-40df-a30e-19ed0c34f9b2"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:31:32.380076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:32.380867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:32.381475 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:32.394286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] Creating event network-vif-deleted:ba58590d-7cc5-40df-a30e-19ed0c34f9b2 for instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e on hv-cinder-85918 Sep 27 04:31:32.400340 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-0f7de7ea-0bee-49b5-b8c9-df2eccdc344d req-493c9c0a-f12d-46e9-b56d-97873d6197d4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121294 Sep 27 04:31:32.400949 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 455/922] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:31:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:33.164083 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:33.164917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:33.171772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:33.172273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:33.172719 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:33.173429 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:33.173935 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:33.174361 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:33.189142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e HTTP/1.1 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:33.190892 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:33.193107 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'e293adc2-d0ac-46d7-92a9-a61e55ca2f5e'} Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:33.194018 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:33.195306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:33.196459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:33.196459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e1a03e7d-eac7-441f-8c9c-0cb9a927912f tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 404 len: 111 microversion: 2.1 time: 0.039587 Sep 27 04:31:33.196459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 468/923] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:33 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:33.202267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-26d19a19-de41-4516-a4f1-cf251e13c471 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '.version_select at 0x7fce708b23a0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:33.222613 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-26d19a19-de41-4516-a4f1-cf251e13c471 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-687461414" status: 202 len: 0 microversion: 2.1 time: 0.024496 Sep 27 04:31:33.223198 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 456/924] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 04:31:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-687461414 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:31:34.978885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:34.979386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:34.985717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:34.986233 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:34.986670 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:34.987354 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:34.987808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:34.988236 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:35.003395 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e HTTP/1.1 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:35.004993 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.006597 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'e293adc2-d0ac-46d7-92a9-a61e55ca2f5e'} Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:35.007571 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.009104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.010759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:35.010759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f2fb973a-f0e9-4019-98e0-33e1f377f49a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 404 len: 111 microversion: 2.1 time: 0.031348 Sep 27 04:31:35.010759 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 469/925] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:31:34 2022] DELETE /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:35.015194 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:35.015933 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: e293adc2-d0ac-46d7-92a9-a61e55ca2f5e] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:35.022446 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:35.022937 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:35.023378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:35.024068 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:35.024531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:35.024959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:35.038517 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e HTTP/1.1 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:31:35.040109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.041674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'e293adc2-d0ac-46d7-92a9-a61e55ca2f5e'} Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:35.042590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. Sep 27 04:31:35.044077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:35.045497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance e293adc2-d0ac-46d7-92a9-a61e55ca2f5e could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:35.045497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-57db5aa6-b1ac-403a-9f52-37a384c1f488 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e" status: 404 len: 111 microversion: 2.1 time: 0.030555 Sep 27 04:31:35.045497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 457/926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:35 2022] GET /compute/v2.1/servers/e293adc2-d0ac-46d7-92a9-a61e55ca2f5e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:31:36.177185 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:36.177712 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:36.184055 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.184538 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.184976 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.185661 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.186152 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.186577 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.200350 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 HTTP/1.1 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:36.201912 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.203583 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'af03ef64-1418-4b7a-a0ee-1294cedb21a1'} Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:36.205172 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.206902 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.209142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:36.209142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-ddf3bfed-55b1-4b6f-9a87-930c97c11d28 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 404 len: 111 microversion: 2.1 time: 0.030609 Sep 27 04:31:36.209142 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 470/927] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:31:36 2022] DELETE /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:36.212278 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:36.212900 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: af03ef64-1418-4b7a-a0ee-1294cedb21a1] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:36.219444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.219936 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.220383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.221073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.221531 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.221998 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.237149 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 HTTP/1.1 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:31:36.238687 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.240839 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'af03ef64-1418-4b7a-a0ee-1294cedb21a1'} Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:36.241789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. Sep 27 04:31:36.243263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.244565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance af03ef64-1418-4b7a-a0ee-1294cedb21a1 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:36.244565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6c94fab8-fc6f-45f3-9c5e-ef9f226cbf04 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1" status: 404 len: 111 microversion: 2.1 time: 0.032151 Sep 27 04:31:36.244565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 458/928] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:36 2022] GET /compute/v2.1/servers/af03ef64-1418-4b7a-a0ee-1294cedb21a1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:36.252733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:36.253239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:36.259768 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.260253 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.260698 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.261386 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.261957 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.262394 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.285223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b HTTP/1.1 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:36.286829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.288415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '7842fec9-9ef4-47fb-b9ff-987b5ba3468b'} Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:36.289326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.290950 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.292526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:36.292526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d177fc9c-2de5-4c4e-9e38-a0e8e8286934 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 404 len: 111 microversion: 2.1 time: 0.039417 Sep 27 04:31:36.292526 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 471/929] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:31:36 2022] DELETE /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:36.296672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:36.297286 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 7842fec9-9ef4-47fb-b9ff-987b5ba3468b] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:36.303711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.304199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.304634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.305318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:36.305845 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:36.306281 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:36.320580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b HTTP/1.1 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:31:36.322122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.323705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '7842fec9-9ef4-47fb-b9ff-987b5ba3468b'} Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:36.324679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. Sep 27 04:31:36.326152 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:36.327468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 7842fec9-9ef4-47fb-b9ff-987b5ba3468b could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:36.327468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e9f51b6c-13ed-45c6-b221-fae4ed070b14 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b" status: 404 len: 111 microversion: 2.1 time: 0.030710 Sep 27 04:31:36.327468 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 459/930] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:36 2022] GET /compute/v2.1/servers/7842fec9-9ef4-47fb-b9ff-987b5ba3468b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:37.464878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:37.465367 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:37.471693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:37.472180 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:37.472619 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:37.473307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:37.473787 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:37.474217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:37.487345 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 HTTP/1.1 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:37.489411 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.491071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '93c1b78a-1f68-4d8b-a00a-15ed4dfe4779'} Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:37.491870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.493093 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.494432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:37.494432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-72397025-2cad-4077-9246-2f9fc921567a tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 404 len: 111 microversion: 2.1 time: 0.029778 Sep 27 04:31:37.494432 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 472/931] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:31:37 2022] DELETE /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:37.499055 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:37.499669 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] [instance: 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:31:37.506779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:37.507265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:37.507705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:37.508391 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:31:37.508850 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:31:37.509276 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:31:37.522873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] HTTP exception thrown: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Request method failure captured: Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 HTTP/1.1 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnuiHgUyH1Uxw_hWnmLYQwqx4usin2oltdZH0kZ9KmUd-lR1TznPWDzkPjNiJ_6m5x0JoKlXNt7s_s5Y7Zc9Z9dxVwp4u1oJzVzltExzhrTfLz91NUQr3LuAoVtr6UTosnGp6saVJbOdXNpf6BCDLts2jerrSLhP_5OK-Anwr6-3Vrk Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:31:37.524374 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d7c74ffec54740eca3cd6b707f1cb732", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d7c74ffec54740eca3cd6b707f1cb732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: d7c74ffec54740eca3cd6b707f1cb732 Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-139081326 Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: dfb35bfba72546f3853b9dd9ef7f5078 Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-AttachVolumeNegativeTest-139081326-project Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.525995 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '93c1b78a-1f68-4d8b-a00a-15ed4dfe4779'} Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:37.526988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. Sep 27 04:31:37.528464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:31:37.530079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] Returning 404 to user: Instance 93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:37.530079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-490ae9d6-9315-4f68-8f9e-be4a2fdbb007 tempest-AttachVolumeNegativeTest-139081326 tempest-AttachVolumeNegativeTest-139081326-project] 10.222.0.26 "GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779" status: 404 len: 111 microversion: 2.1 time: 0.030502 Sep 27 04:31:37.530079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 460/932] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:31:37 2022] GET /compute/v2.1/servers/93c1b78a-1f68-4d8b-a00a-15ed4dfe4779 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:31:42.000741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e63fe557-2f94-452d-8d9c-29aa1badef2e tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Action: 'create', calling method: .version_select at 0x7fce70a5caf0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-714146694", "metadata": {"Type": "work"}}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:31:42.294274 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e63fe557-2f94-452d-8d9c-29aa1badef2e tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.409392 Sep 27 04:31:42.295077 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 473/933] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 04:31:41 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 411 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 27 04:31:42.308135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-62c0ffed-d683-47c4-82ba-38a52858e6ac tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce70f93310>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:42.357510 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-62c0ffed-d683-47c4-82ba-38a52858e6ac tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 200 len: 352 microversion: 2.1 time: 0.053384 Sep 27 04:31:42.358129 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 461/934] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:31:42 2022] GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 352 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:43.372310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6fc53e65-e219-4401-ae4b-ddddefed3ee5 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce707bd1f0>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:43.421496 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6fc53e65-e219-4401-ae4b-ddddefed3ee5 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 200 len: 353 microversion: 2.1 time: 0.053496 Sep 27 04:31:43.422102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 474/935] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:31:43 2022] GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 353 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:43.435710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-77dcd797-562d-478a-9371-430313f1346e tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce70a33b80>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:43.483018 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-77dcd797-562d-478a-9371-430313f1346e tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 200 len: 353 microversion: 2.1 time: 0.051479 Sep 27 04:31:43.483567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 462/936] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:31:43 2022] GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 353 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:43.503561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7fbe3763-6697-431d-a62c-396b604b05b1 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce70a10700>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:43.563250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7fbe3763-6697-431d-a62c-396b604b05b1 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 202 len: 0 microversion: 2.1 time: 0.063455 Sep 27 04:31:43.563841 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 475/937] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 04:31:43 2022] DELETE /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 0 bytes in 65 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 04:31:43.571910 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7c365f71-7fd9-4c5e-871e-2e84feb93707 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce7084ddc0>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:43.619561 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7c365f71-7fd9-4c5e-871e-2e84feb93707 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 200 len: 352 microversion: 2.1 time: 0.051454 Sep 27 04:31:43.620150 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 463/938] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:31:43 2022] GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 352 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:31:44.634791 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0eab6789-4490-45ca-abb2-4ea50476fb51 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Calling method '.version_select at 0x7fce70a63430>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:31:44.662604 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-0eab6789-4490-45ca-abb2-4ea50476fb51 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] HTTP exception thrown: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0eab6789-4490-45ca-abb2-4ea50476fb51 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Request method failure captured: Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 HTTP/1.1 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnytv1FJowSJoKejuNRaq0UNyn0CVwrmISRghqY2QiX6zhUEGSzRMgt4QuT7ut72v2CApUpWcYaREHgV7r8uCarj9vjsVzW7dvKN_AZyZ2tG3TzxgF9VwhmZV5faZaRg2HPFQIuFf9n1IBqWQy664jYmrI4g3plupLkA3gtiGQTrPi8 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: b86c8a654e564d8fbe35c070363144f1 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesGetTestJSON-655613078 Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:31:44.664564 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b86c8a654e564d8fbe35c070363144f1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b86c8a654e564d8fbe35c070363144f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b86c8a654e564d8fbe35c070363144f1", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b86c8a654e564d8fbe35c070363144f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesGetTestJSON-655613078 Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: b86c8a654e564d8fbe35c070363144f1 Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesGetTestJSON-655613078 Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesGetTestJSON-655613078-project Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 3bb2b5e770924b7383739a66e9308638 Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesGetTestJSON-655613078-project Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': 'd68650e5-8a43-4764-81b0-f98a6859f6a4'} Sep 27 04:31:44.666260 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. (HTTP 404) (Request-ID: req-d90e447f-4c45-4e47-8d8a-3b1d913c5a8c) Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:44.667201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 04:31:44.668643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 04:31:44.670235 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0eab6789-4490-45ca-abb2-4ea50476fb51 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] Returning 404 to user: Volume d68650e5-8a43-4764-81b0-f98a6859f6a4 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0eab6789-4490-45ca-abb2-4ea50476fb51 tempest-VolumesGetTestJSON-655613078 tempest-VolumesGetTestJSON-655613078-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4" status: 404 len: 109 microversion: 2.1 time: 0.035814 Sep 27 04:31:44.671690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 476/939] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 04:31:44 2022] GET /compute/v2.1/os-volumes/d68650e5-8a43-4764-81b0-f98a6859f6a4 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:32:10.351387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:10.352157 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=47,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:10.356996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:10.358022 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:10.367889 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:10.368378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:10.369104 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:10.369819 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:10.370388 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:10.370842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:10.398033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:32:10.485176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-ddd9c753-e557-4d65-b341-c6c2c689a6d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "DELETE /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 204 len: 0 microversion: 2.1 time: 0.135114 Sep 27 04:32:10.488781 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 464/940] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:32:10 2022] DELETE /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:32:10.492163 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=20 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:10.493384 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:10.500594 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:10.501251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:10.509258 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:10.509767 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:10.510210 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:10.510916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:10.511369 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:10.511791 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:10.663711 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:10.677699 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d4c15379-5f51-4714-9735-043a23925520 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1577 microversion: 2.1 time: 0.187087 Sep 27 04:32:10.678328 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 477/941] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:10 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1577 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:32:11.701223 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:11.702116 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:11.708640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:11.709138 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:11.709575 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:11.710480 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:11.710938 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:11.711361 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:11.848673 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:11.856856 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b81829f4-4fec-431c-aaf5-fa8a0dade446 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1577 microversion: 2.1 time: 0.159888 Sep 27 04:32:11.858255 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 465/942] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:11 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1577 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:12.882852 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:12.883534 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:12.890520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:12.891012 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:12.891454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:12.892353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:12.892816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:12.893243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:12.984436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:12.992716 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-c1e28422-d321-4426-9af9-214b81a51698 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 200 len: 1509 microversion: 2.1 time: 0.113858 Sep 27 04:32:12.993334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 478/943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:12 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 1509 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:32:13.095312 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "298d0196-0c87-4bfd-a862-be6cbe3a28f5", "name": "network-vif-deleted", "tag": "1712560b-01e8-4b96-8c66-8877d22eba22"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:13.105088 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:13.105649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:13.106209 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:13.117800 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] Creating event network-vif-deleted:1712560b-01e8-4b96-8c66-8877d22eba22 for instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 on hv-cinder-85918 Sep 27 04:32:13.123487 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-cac5c692-b873-49d6-a925-49a4caef3d72 req-e4d19841-b04f-4212-a96a-7ea0f4559125 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031343 Sep 27 04:32:13.124322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 466/944] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:32:13.559144 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-200430580", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "networks": [{"uuid": "2d8e1010-a99b-4d47-a52c-48fef59dac1e"}]}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:13.623415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] validate_networks() for [('2d8e1010-a99b-4d47-a52c-48fef59dac1e', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:32:13.984570 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:32:13.985198 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:32:13.985462 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:32:13.985909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:32:13.987038 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:32:13.998145 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:13.998641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:13.999276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.000182 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:14.000642 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:14.001307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.018943 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:14.019788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:14.034639 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:14.035434 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:14.035924 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.036652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:14.038226 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:14.038667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.042245 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Getting quotas for project 494bdfe04cfa4dfab2c7ab389a889298. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:32:14.050054 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Getting quotas for user 20bf50d405e84ca7abd123e3c7a98c69 and project 494bdfe04cfa4dfab2c7ab389a889298. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:32:14.064757 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:32:14.067114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:32:14.071054 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] HTTP exception thrown: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Request method failure captured: Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 HTTP/1.1 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnwsQBwYHnQ_PGyzlt_S6KBiJbFTHyTPilJAkq2AL5BQ77owISesAEMfIqIUR3FWdXUwaEDuLEcrqIBmDSEKYiZ3xSNW60INcigHq0jDH-I5XcgoLn1v1Ij32MUqYlASpSbDpmFwsA2VXFwj-ibq_oL1l2G9DWZUjgTpmX8kc1Yf6w4 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:32:14.073955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_53d86b6e8a8343fca03ca6d38200f23f", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-TestVolumeBootPattern-57324624 Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 44a710d801334b20b9f433eaad61cbc6 Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:32:14.076145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '298d0196-0c87-4bfd-a862-be6cbe3a28f5'} Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:32:14.076975 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:32:14.079093 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:14.082501 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Returning 404 to user: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:32:14.083404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-87efb3be-5045-4719-9090-eda270eed3ac tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 404 len: 111 microversion: 2.1 time: 0.070974 Sep 27 04:32:14.083969 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 467/945] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:14 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:32:14.128276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-4be78a55-5f4a-445d-86eb-829a66eb1a4e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.572933 Sep 27 04:32:14.129138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 479/946] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:32:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 575 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:32:14.142464 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-759afa24-3f7d-4102-97db-1b116fa97c22 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:14.144674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-759afa24-3f7d-4102-97db-1b116fa97c22 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:14.207755 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-759afa24-3f7d-4102-97db-1b116fa97c22 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:14.210513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1852248813", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1258945985", "security_groups": [{"name": "tempest-secgroup-smoke-1621255540"}], "block_device_mapping_v2": [{"uuid": "c535295c-02ac-420a-8d45-3ae388060e4f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "df4f6840-004b-4387-a687-683e2895186f"}]}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:14.214955 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-759afa24-3f7d-4102-97db-1b116fa97c22 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1213 microversion: 2.1 time: 0.078375 Sep 27 04:32:14.230167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 468/947] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:14 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1213 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:14.330259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] validate_networks() for [('df4f6840-004b-4387-a687-683e2895186f', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:32:14.710032 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:32:14.710454 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:32:14.710928 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:32:14.711331 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:32:14.711737 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:32:14.721139 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:14.721676 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:14.722184 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.722944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:14.723395 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:14.723816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:14.740308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:32:14.744007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for user 44a710d801334b20b9f433eaad61cbc6 and project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:32:14.809353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:32:14.812115 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c535295c-02ac-420a-8d45-3ae388060e4f',volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:32:14.947102 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-69580fa4-be17-409b-94c6-b2daeb01bf0a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.742500 Sep 27 04:32:14.947965 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 480/948] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:32:14 2022] POST /compute/v2.1/servers => generated 406 bytes in 745 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:32:14.959508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-81e7379c-c340-4e0b-8427-c525a95f473e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:14.960220 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-81e7379c-c340-4e0b-8427-c525a95f473e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:15.014707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-81e7379c-c340-4e0b-8427-c525a95f473e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:15.021282 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-81e7379c-c340-4e0b-8427-c525a95f473e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1095 microversion: 2.1 time: 0.066191 Sep 27 04:32:15.021887 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 469/949] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:14 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1095 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:15.240472 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:15.241095 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:15.252968 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:15.253461 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:15.253936 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:15.254641 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:15.255104 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:15.255531 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:15.346071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:15.355019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-8513f5db-9ca6-4757-8832-095d052dc0ab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1265 microversion: 2.1 time: 0.118335 Sep 27 04:32:15.355872 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 481/950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:15 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1265 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:16.044647 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:16.045267 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:16.051869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:16.052570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:16.053024 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:16.053786 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:16.054256 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:16.054868 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:16.127409 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:16.135339 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e3c50ccc-edf8-424e-889c-e0bce3beb0d1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1193 microversion: 2.1 time: 0.094737 Sep 27 04:32:16.135911 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 470/951] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:16 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1193 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:16.379335 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:16.380088 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:16.386539 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:16.387087 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:16.387568 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:16.388273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:16.388732 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:16.389157 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:16.679181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:16.688422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-7491c447-21d9-41da-ad3f-56d724b5582e tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.312779 Sep 27 04:32:16.688870 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 482/952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:16 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:17.159847 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:17.161226 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:17.170656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:17.171234 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:17.171879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:17.172593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:17.173062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:17.173491 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:17.239994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:17.248075 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2eb01510-f58a-4e77-a24a-6a4848d39812 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1199 microversion: 2.1 time: 0.092267 Sep 27 04:32:17.248657 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 471/953] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:17 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1199 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:17.712373 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:17.712955 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:17.719175 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:17.719666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:17.720106 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:17.720808 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:17.722433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:17.722433 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:17.731582 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb3757a7-2228-48e1-ad30-95f66c6b55f8", "tag": "0a8c54ff-8ed7-4f2c-9293-48d6c353b576"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:17.746417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:17.753979 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:17.754684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:17.768683 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] Creating event network-changed:0a8c54ff-8ed7-4f2c-9293-48d6c353b576 for instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 on hv-cinder-85918 Sep 27 04:32:17.773904 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-eb595d6d-d7b2-4f72-b8a0-d0867d728115 req-791b8c10-b233-44bb-884c-0c8cbc755881 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046355 Sep 27 04:32:17.774655 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 472/954] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:32:18.110216 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:18.117982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3e8c08e5-b8c8-4fc9-807b-b4652c240dec tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.409684 Sep 27 04:32:18.118689 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 483/955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:17 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:18.273866 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:18.274564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:18.280621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:18.281112 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:18.281547 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:18.282265 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:18.282722 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:18.283151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:18.413570 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:18.422483 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-156bc423-fd60-4045-879a-f8b67b425b55 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.154589 Sep 27 04:32:18.423168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 473/956] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:18 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:19.142004 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:19.142561 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:19.149166 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:19.149666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:19.150123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:19.150813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:19.151261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:19.151680 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:19.281573 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:19.290784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a25db5e3-da54-4835-82e1-3a3e41bc586a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.153791 Sep 27 04:32:19.291342 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 484/957] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:19 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:19.446008 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:19.446634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:19.452985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:19.453484 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:19.453971 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:19.454675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:19.455141 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:19.455577 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:19.587287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:19.610996 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-65b403ad-9e8d-42ff-9325-c807e3e8a6ab tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.168898 Sep 27 04:32:19.611586 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 474/958] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:19 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:19.796897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56108588-5052-4cbc-b248-29229aef7aec", "tag": "f9768712-2949-4085-b485-9a6386391423"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:19.806163 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:19.806651 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:19.807079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:19.817991 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] Creating event network-changed:f9768712-2949-4085-b485-9a6386391423 for instance 56108588-5052-4cbc-b248-29229aef7aec on hv-cinder-85918 Sep 27 04:32:19.822904 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-3ee239a1-17ab-4209-bc81-25b980d6cbfc req-9b5c8536-792c-4a32-8d49-5ef8a68a864d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030384 Sep 27 04:32:19.823484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 485/959] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:20.314604 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:20.315485 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:20.322355 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:20.322946 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:20.323459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:20.324412 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:20.325785 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:20.326224 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:20.470081 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:20.477559 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-70a3e6d1-f346-4592-aad3-56d3043d1579 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.167035 Sep 27 04:32:20.478139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 475/960] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:20 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:20.633494 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:20.634364 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:20.641788 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:20.642284 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:20.642723 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:20.643423 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:20.643913 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:20.644343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:20.832519 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:20.840259 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-76bb3700-48c8-4502-82bd-985657a87219 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.211020 Sep 27 04:32:20.840813 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 486/961] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:20 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:21.500368 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:21.501010 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:21.507635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:21.508123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:21.508565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:21.509251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:21.509710 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:21.510174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:21.639176 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:21.647070 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-c2138f79-bc23-4854-8f27-6891a7e14b76 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.150572 Sep 27 04:32:21.647842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 476/962] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:21 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:21.862885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:21.863484 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:21.869916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:21.870405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:21.870853 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:21.872059 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:21.872520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:21.872946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:22.002188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:22.009803 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-058a682e-c919-4d2a-b587-a67ee1487b45 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.150779 Sep 27 04:32:22.010366 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 487/963] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:21 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:22.670360 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:22.670951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:22.677184 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:22.677671 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:22.678147 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:22.678831 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:22.679287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:22.679711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:22.816387 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:22.824779 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-b540ca8e-a40b-4fe0-a406-26737c17e5a1 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.157725 Sep 27 04:32:22.825350 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 477/964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:22 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:23.032410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:23.033027 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:23.039424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:23.039908 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:23.040348 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:23.041035 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:23.041491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:23.041949 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:23.174867 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:23.182509 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1b636722-06f2-4e3b-95b9-51c3753f47c1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.153948 Sep 27 04:32:23.183071 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 488/965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:23 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:23.438747 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb3757a7-2228-48e1-ad30-95f66c6b55f8", "name": "network-vif-plugged", "status": "completed", "tag": "0a8c54ff-8ed7-4f2c-9293-48d6c353b576"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:23.447888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:23.448366 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:23.448815 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:23.459327 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] Creating event network-vif-plugged:0a8c54ff-8ed7-4f2c-9293-48d6c353b576 for instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 on hv-cinder-85918 Sep 27 04:32:23.464279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-c8cfdd1b-a5ef-4358-81ca-b01628656b66 req-5234dcb6-86fa-4d47-80e5-574719ee05a3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.148170 Sep 27 04:32:23.465352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 478/966] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:23.847889 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:23.848520 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:23.854399 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:23.854881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:23.855313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:23.855990 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:23.856438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:23.856856 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:23.984188 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:23.991663 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5c7fbcfd-db13-45d0-a207-623507fd6770 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.147620 Sep 27 04:32:23.992220 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 489/967] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:23 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:24.206966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:24.207633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:24.214920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:24.215423 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:24.215867 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:24.216568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:24.217033 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:24.217465 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:24.347037 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:24.354603 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-49e86fff-be4f-4f53-8441-de563841a7e9 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.152543 Sep 27 04:32:24.355248 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 479/968] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:24 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:25.014823 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:25.015435 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:25.017948 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=4 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:25.018590 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=12 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:25.024944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:25.025424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:25.025893 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:25.026574 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:25.027025 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:25.027449 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:25.159262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:25.166930 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5cd5634a-0a01-46e5-abe2-72d884e13e5a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1313 microversion: 2.1 time: 0.156423 Sep 27 04:32:25.168165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 490/969] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:25 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:25.377870 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:25.378492 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:25.380417 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=4 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:25.386078 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:25.386609 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:25.387052 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:25.387739 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:25.388200 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:25.388631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:25.521572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:25.529675 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8586de8d-8015-4f7d-9779-2a4f37bb2e58 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.155713 Sep 27 04:32:25.530260 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 480/970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:25 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:26.193875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:26.194497 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:26.201438 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.201953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.202397 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.203088 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.203548 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.203978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.334635 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:26.342062 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5994d02b-b2e0-4f1e-8480-7c8fb0473d6f tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1498 microversion: 2.1 time: 0.152134 Sep 27 04:32:26.342631 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 491/971] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:26 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1498 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:26.552459 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:26.553083 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:26.560061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.560551 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.560998 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.561719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.562208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.562640 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.697565 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:26.705580 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-16377302-31b0-4c71-80e8-a7ab10674295 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.157083 Sep 27 04:32:26.706172 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 481/972] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:26 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:26.773543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:26.774014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:26.779917 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.780398 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.780833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.781513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.781996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.782421 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.807127 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:32:26.879070 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e64f57e3-bcef-4683-affa-5e5dba3bc700 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "DELETE /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 204 len: 0 microversion: 2.1 time: 0.109346 Sep 27 04:32:26.879874 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 492/973] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:32:26 2022] DELETE /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:32:26.888337 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:26.888951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:26.894862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.895350 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.895789 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:26.896738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:26.897197 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:26.897626 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:27.024646 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:27.032813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f056609b-5354-45a6-9fbf-b50754f626a3 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1504 microversion: 2.1 time: 0.148440 Sep 27 04:32:27.033370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 482/974] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:26 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1504 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:27.729022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:27.729829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:27.736135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:27.736618 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:27.737048 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:27.737733 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:27.738205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:27.738624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:27.872181 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:27.875720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:27.881293 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-515e4534-c29b-4dcd-9b3c-a1e434365996 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.156574 Sep 27 04:32:27.881916 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 493/975] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:27 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:32:28.056287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:28.056902 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:28.063784 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:28.064272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:28.064715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:28.065404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:28.065894 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:28.066322 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:28.084544 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:28.195067 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:28.202634 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-bcd952a1-ecdd-4841-a491-44219d4d1c9c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1504 microversion: 2.1 time: 0.150203 Sep 27 04:32:28.203199 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 483/976] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:28 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1504 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:28.766261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb3757a7-2228-48e1-ad30-95f66c6b55f8", "name": "network-vif-deleted", "tag": "0a8c54ff-8ed7-4f2c-9293-48d6c353b576"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:28.779706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:28.780448 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:28.780958 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:28.793330 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] Creating event network-vif-deleted:0a8c54ff-8ed7-4f2c-9293-48d6c353b576 for instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 on hv-cinder-85918 Sep 27 04:32:28.798888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-2f58e591-b361-4a57-8d0f-7d553ff1bd1e req-d7732c7a-2f5f-46ec-a300-0180311215b3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036921 Sep 27 04:32:28.799475 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 494/977] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:32:28.905117 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:28.905734 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:28.911744 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:28.912242 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:28.912685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:28.913378 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:28.913873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:28.914305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:29.049873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:29.057461 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6513563e-d1f4-4391-b446-2390b27db740 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.156766 Sep 27 04:32:29.058094 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 484/978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:28 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:29.226262 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:29.226971 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:29.233256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:29.233766 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:29.234201 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:29.234945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:29.235396 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:29.235816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:29.272320 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:29.272828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:29.273643 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:29.273797 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lazy-loading 'fault' on Instance uuid cb3757a7-2228-48e1-ad30-95f66c6b55f8 {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:32:29.320405 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:29.328861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cf3c6e25-0aaa-4772-9605-5e39f35a7083 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 200 len: 1303 microversion: 2.1 time: 0.106957 Sep 27 04:32:29.329703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 495/979] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:29 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 1303 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:29.369457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56108588-5052-4cbc-b248-29229aef7aec", "name": "network-vif-plugged", "status": "completed", "tag": "f9768712-2949-4085-b485-9a6386391423"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:29.380308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:29.380795 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:29.381229 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:29.395254 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] Creating event network-vif-plugged:f9768712-2949-4085-b485-9a6386391423 for instance 56108588-5052-4cbc-b248-29229aef7aec on hv-cinder-85918 Sep 27 04:32:29.400810 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-2daf8202-9f55-4105-935a-7af652687faf req-fee469af-df66-4f27-b23b-f02767508bea service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035095 Sep 27 04:32:29.402076 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 485/980] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:32:30.080060 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:30.080679 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:30.087217 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.087704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.088146 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.089347 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.089838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.090272 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.223082 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:30.231465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-6950eaae-678b-4d90-a69e-2ff2a845af5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1267 microversion: 2.1 time: 0.155291 Sep 27 04:32:30.232034 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 496/981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:30 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:30.353051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:30.353700 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: cb3757a7-2228-48e1-ad30-95f66c6b55f8] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:30.360469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.360957 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.361397 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.362110 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.362574 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.363003 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.375879 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] HTTP exception thrown: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Request method failure captured: Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 HTTP/1.1 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnzLc29MiteefgGv03N3M1mD8GHZ5ZMpDVnRZBJGNvhgoBNca29nyiRnlCkG7-1s71anMFg0LNF3MOgNcMFMPQVV6iT3_9HUMbLWl7GOGLwLnlr9GOPir9T6bJmF9UuVzjrlB-ZtUJeBmx3f7tW1EhXALHmitj_QmxpJ074oZm7buVU Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 494bdfe04cfa4dfab2c7ab389a889298 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-VolumesActionsTest-874250999 Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:32:30.377469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_494bdfe04cfa4dfab2c7ab389a889298", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-VolumesActionsTest-874250999 Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 494bdfe04cfa4dfab2c7ab389a889298 Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-VolumesActionsTest-874250999 Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-VolumesActionsTest-874250999-project Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 20bf50d405e84ca7abd123e3c7a98c69 Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-VolumesActionsTest-874250999-project Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. Sep 27 04:32:30.379167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'cb3757a7-2228-48e1-ad30-95f66c6b55f8'} Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:32:30.380170 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. Sep 27 04:32:30.381712 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:32:30.383190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Returning 404 to user: Instance cb3757a7-2228-48e1-ad30-95f66c6b55f8 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:32:30.383190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-17c77449-d9b5-4c7f-91c5-51275b0eee98 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8" status: 404 len: 111 microversion: 2.1 time: 0.030189 Sep 27 04:32:30.383190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 486/982] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:30 2022] GET /compute/v2.1/servers/cb3757a7-2228-48e1-ad30-95f66c6b55f8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:32:30.428710 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-621059756", "imageRef": "96a21f47-e58a-49ec-acfe-076e37ceafc8", "flavorRef": "452", "networks": [{"uuid": "2d8e1010-a99b-4d47-a52c-48fef59dac1e"}]}} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:30.486895 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.network.neutron [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] validate_networks() for [('2d8e1010-a99b-4d47-a52c-48fef59dac1e', None, None, None, None, None)] {{(pid=120423) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:32:30.824513 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Flavor limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:32:30.824934 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Image limits 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:32:30.825404 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Flavor pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:32:30.825838 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Image pref 0:0:0 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:32:30.826251 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.virt.hardware [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120423) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:32:30.834897 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.835375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.835806 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.836525 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:30.836981 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:30.837406 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:30.853685 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Getting quotas for project 494bdfe04cfa4dfab2c7ab389a889298. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:32:30.857691 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.quota [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Getting quotas for user 20bf50d405e84ca7abd123e3c7a98c69 and project 494bdfe04cfa4dfab2c7ab389a889298. Resources: {'instances', 'cores', 'ram'} {{(pid=120423) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:32:30.868243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Going to run 1 instances... {{(pid=120423) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:32:30.871002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='96a21f47-e58a-49ec-acfe-076e37ceafc8',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120423) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:32:30.917826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-76d34956-5766-4046-abfa-a25800a4de89 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.492935 Sep 27 04:32:30.918463 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 497/983] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:32:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 494 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:32:30.929415 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-2b3006c6-bd0c-47e6-a96e-d5d0ca1414f5 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:30.930163 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-2b3006c6-bd0c-47e6-a96e-d5d0ca1414f5 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:30.988716 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-2b3006c6-bd0c-47e6-a96e-d5d0ca1414f5 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:30.995976 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-2b3006c6-bd0c-47e6-a96e-d5d0ca1414f5 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1213 microversion: 2.1 time: 0.070417 Sep 27 04:32:30.996726 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 487/984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:30 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1213 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:31.254174 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:31.254828 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:31.262165 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:31.262667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:31.263109 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:31.263812 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:31.264270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:31.264702 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:31.400771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:31.409276 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a276388a-da67-4197-9bc9-ec78ede12a22 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1454 microversion: 2.1 time: 0.159012 Sep 27 04:32:31.409888 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 498/985] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:31 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1454 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:31.432893 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:31.433615 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:31.440396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:31.440959 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:31.441476 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:31.442257 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:31.442788 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:31.443400 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:31.586469 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:31.594546 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-532a0886-c18d-490c-b7f9-c5f64842813a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1454 microversion: 2.1 time: 0.166105 Sep 27 04:32:31.595162 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 488/986] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:31 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1454 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:32.022780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:32.023964 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:32.032400 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:32.033223 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:32.034464 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:32.035422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:32.036100 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:32.036745 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:32.139254 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:32.148230 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-1f69524f-3f51-49cb-8ac5-c07efff07d9a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1283 microversion: 2.1 time: 0.132834 Sep 27 04:32:32.149400 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 499/987] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:32 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1283 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:33.172294 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:33.173590 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:33.181314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:33.182617 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:33.183353 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:33.184246 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:33.184874 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:33.185314 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:33.334212 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "098cd137-f2d7-47ab-ae56-86245494f738", "tag": "62eec92a-875e-4b18-93d9-31fe99c92ac4"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:33.343758 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9,nova_api:UPDATE=1 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:33.350014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:33.359930 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:33.361011 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:33.369091 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:33.378135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-f509947a-e59a-4f53-8acd-eb210994be1a tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1313 microversion: 2.1 time: 0.209950 Sep 27 04:32:33.389826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] Creating event network-changed:62eec92a-875e-4b18-93d9-31fe99c92ac4 for instance 098cd137-f2d7-47ab-ae56-86245494f738 on hv-cinder-85918 Sep 27 04:32:33.389826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-aedd99be-201a-41a4-8acf-d0037d61c1fa req-2f4aa1bc-5424-4759-9772-077329ab4286 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056087 Sep 27 04:32:33.389826 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 500/988] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:33.390513 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 489/989] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:33 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1313 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:33.598241 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56108588-5052-4cbc-b248-29229aef7aec", "tag": "f9768712-2949-4085-b485-9a6386391423"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:33.609306 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:33.609945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:33.610377 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:33.625410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] Creating event network-changed:f9768712-2949-4085-b485-9a6386391423 for instance 56108588-5052-4cbc-b248-29229aef7aec on hv-cinder-85918 Sep 27 04:32:33.631837 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-db0f765d-8e86-4935-ac4f-2928475aa875 req-0b06d2f0-5b4e-4a62-81b6-2101b5c6464b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102669 Sep 27 04:32:33.633898 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 501/990] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:34.403738 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:34.404352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:34.411765 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:34.412218 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:34.412656 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:34.413349 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:34.413826 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:34.414251 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:34.546684 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:34.555168 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8a9da1db-7880-46b3-9aca-e7f9857fc305 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1313 microversion: 2.1 time: 0.155678 Sep 27 04:32:34.555856 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 490/991] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:34 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:35.579709 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:35.580357 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:35.586522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:35.587006 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:35.587442 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:35.588353 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:35.588982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:35.589598 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:35.718205 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:35.726249 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-47f1d1cf-ddfd-46ed-a3c3-19d13e6b7795 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1313 microversion: 2.1 time: 0.151312 Sep 27 04:32:35.727138 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 502/992] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:35 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1313 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:36.748408 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:36.749021 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:36.755419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:36.755904 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:36.756342 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:36.757025 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:36.757478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:36.757973 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:36.882114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:36.890308 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-dd90a534-1ed6-41a4-b084-2eff05be122d tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1313 microversion: 2.1 time: 0.145638 Sep 27 04:32:36.890869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 491/993] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:36 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1313 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:37.412061 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "098cd137-f2d7-47ab-ae56-86245494f738", "name": "network-vif-plugged", "status": "completed", "tag": "62eec92a-875e-4b18-93d9-31fe99c92ac4"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:37.422313 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:37.423039 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:37.423474 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:37.436690 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] Creating event network-vif-plugged:62eec92a-875e-4b18-93d9-31fe99c92ac4 for instance 098cd137-f2d7-47ab-ae56-86245494f738 on hv-cinder-85918 Sep 27 04:32:37.441601 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-6d691ebb-2010-43cc-9c28-329add216d2a req-b8b961f9-146b-44a8-80c4-5c62cd9f2952 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033706 Sep 27 04:32:37.442243 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 503/994] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:37.912479 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:37.913041 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:37.919403 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:37.919882 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:37.920321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:37.921006 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:37.921457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:37.921916 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:38.056705 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:38.064432 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-3051bcfd-a23b-43e1-a77f-212ef62d8bab tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1313 microversion: 2.1 time: 0.155905 Sep 27 04:32:38.064988 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 492/995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:37 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:39.088309 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:39.089003 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:39.095529 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.096008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.096440 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.097123 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.097574 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.098036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.225309 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:39.233505 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-219034bc-9d2f-4d50-801f-c2193534ab56 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1497 microversion: 2.1 time: 0.149898 Sep 27 04:32:39.234101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 504/996] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:39 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:39.674522 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:39.675042 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:39.681321 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.681840 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.682283 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.682964 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.683419 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.683842 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.708085 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Going to try to terminate instance {{(pid=120424) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:32:39.782380 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-05aca9c1-9780-4489-b7a5-7ac9fc437e10 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "DELETE /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 204 len: 0 microversion: 2.1 time: 0.111454 Sep 27 04:32:39.783061 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 493/997] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:32:39 2022] DELETE /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:32:39.791861 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:39.792477 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:39.798445 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.798953 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.799380 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.800074 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:39.800536 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:39.800965 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:39.931667 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:39.939224 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-32844df3-8a27-4143-8f8c-4f7fb4fdd7ad tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1503 microversion: 2.1 time: 0.151262 Sep 27 04:32:39.939780 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 505/998] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:39 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1503 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:40.963478 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:40.964211 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:40.966672 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:40.975370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:40.975860 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:40.976301 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:40.977002 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:40.978145 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:40.978638 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:40.995020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:41.112877 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:41.121535 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-10fcbb00-6168-4a87-ad0d-92b0698a9fe2 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1503 microversion: 2.1 time: 0.162295 Sep 27 04:32:41.122131 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 494/999] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:40 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1503 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:41.977422 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "098cd137-f2d7-47ab-ae56-86245494f738", "name": "network-vif-deleted", "tag": "62eec92a-875e-4b18-93d9-31fe99c92ac4"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:32:41.984002 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:41.984687 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=6 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:32:41.989491 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:41.990007 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:41.990469 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:42.002722 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] Creating event network-vif-deleted:62eec92a-875e-4b18-93d9-31fe99c92ac4 for instance 098cd137-f2d7-47ab-ae56-86245494f738 on hv-cinder-85918 Sep 27 04:32:42.007389 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-4f8f4745-d415-4419-adf5-d362f7b2a36f req-fc327dca-18b8-42e2-a109-c06bbed92600 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033677 Sep 27 04:32:42.007979 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 506/1000] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:32:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:32:42.144504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:42.145304 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:42.151709 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:42.152272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:42.152777 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:42.153628 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:42.154825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:42.154825 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:42.185537 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:42.186062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:42.186504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:42.186951 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.objects.instance [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lazy-loading 'fault' on Instance uuid 098cd137-f2d7-47ab-ae56-86245494f738 {{(pid=120424) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:32:42.227948 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bf50d405e84ca7abd123e3c7a98c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '494bdfe04cfa4dfab2c7ab389a889298', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:32:42.235564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6ee1d682-a475-4568-97ba-62cf2ae1e397 tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 200 len: 1303 microversion: 2.1 time: 0.095137 Sep 27 04:32:42.236127 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 495/1001] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:42 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 1303 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:32:43.260582 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:32:43.261273 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] [instance: 098cd137-f2d7-47ab-ae56-86245494f738] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:32:43.267392 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:43.267866 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:43.268295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:43.269225 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:32:43.269671 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:32:43.270114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:32:43.283443 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] HTTP exception thrown: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Request method failure captured: Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 HTTP/1.1 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnzLc29MiteefgGv03N3M1mD8GHZ5ZMpDVnRZBJGNvhgoBNca29nyiRnlCkG7-1s71anMFg0LNF3MOgNcMFMPQVV6iT3_9HUMbLWl7GOGLwLnlr9GOPir9T6bJmF9UuVzjrlB-ZtUJeBmx3f7tW1EhXALHmitj_QmxpJ074oZm7buVU Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 494bdfe04cfa4dfab2c7ab389a889298 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-VolumesActionsTest-874250999 Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:32:43.285023 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_494bdfe04cfa4dfab2c7ab389a889298", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/494bdfe04cfa4dfab2c7ab389a889298"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-VolumesActionsTest-874250999 Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 494bdfe04cfa4dfab2c7ab389a889298 Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-VolumesActionsTest-874250999 Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-VolumesActionsTest-874250999-project Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 20bf50d405e84ca7abd123e3c7a98c69 Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-VolumesActionsTest-874250999-project Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. Sep 27 04:32:43.286734 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '098cd137-f2d7-47ab-ae56-86245494f738'} Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:32:43.287674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. Sep 27 04:32:43.289079 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:32:43.290270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] Returning 404 to user: Instance 098cd137-f2d7-47ab-ae56-86245494f738 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:32:43.290270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-a7b0555d-67f5-4bc4-a17b-c22ae9057d7c tempest-VolumesActionsTest-874250999 tempest-VolumesActionsTest-874250999-project] 10.222.0.26 "GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738" status: 404 len: 111 microversion: 2.1 time: 0.031080 Sep 27 04:32:43.290270 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 507/1002] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:32:43 2022] GET /compute/v2.1/servers/098cd137-f2d7-47ab-ae56-86245494f738 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:34:08.836073 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=5 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:08.836814 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:08.838171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 04:34:08.838318 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Tue Sep 27 04:34:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Sep 27 04:34:08.840174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 04:34:08.840305 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Tue Sep 27 04:34:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Sep 27 04:34:08.846284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1677603389", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1258945985", "security_groups": [{"name": "tempest-secgroup-smoke-1621255540"}], "block_device_mapping_v2": [{"uuid": "43dbfac7-b8ea-42c7-8f5c-b2945bc5332b", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "df4f6840-004b-4387-a687-683e2895186f"}]}} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:34:08.966396 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.network.neutron [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] validate_networks() for [('df4f6840-004b-4387-a687-683e2895186f', None, None, None, None, None)] {{(pid=120424) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 04:34:09.448920 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 04:34:09.449579 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image limits 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 04:34:09.451020 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Flavor pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 04:34:09.451891 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Image pref 0:0:0 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 04:34:09.452581 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.virt.hardware [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120424) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 04:34:09.465376 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:09.466875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:09.467585 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:09.468767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:09.469467 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:09.470385 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:09.491719 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 04:34:09.495657 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.quota [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Getting quotas for user 44a710d801334b20b9f433eaad61cbc6 and project 53d86b6e8a8343fca03ca6d38200f23f. Resources: {'instances', 'cores', 'ram'} {{(pid=120424) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 04:34:09.551164 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Going to run 1 instances... {{(pid=120424) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 04:34:09.553279 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='43dbfac7-b8ea-42c7-8f5c-b2945bc5332b',volume_size=None,volume_type=None)] {{(pid=120424) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 04:34:09.691674 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6a5d7af9-a0ad-4c20-ba12-d1000162214e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.857232 Sep 27 04:34:09.692243 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 496/1003] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 04:34:08 2022] POST /compute/v2.1/servers => generated 406 bytes in 859 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 04:34:09.701308 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=14 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:09.702326 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=58 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:09.703316 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 04:34:09.703453 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Tue Sep 27 04:34:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 (ip 10.222.0.26) !!! Sep 27 04:34:09.708190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-153e1f3d-45b8-46a6-800d-501a4c421332 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:09.708805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-153e1f3d-45b8-46a6-800d-501a4c421332 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:09.767863 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-153e1f3d-45b8-46a6-800d-501a4c421332 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:09.775556 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-153e1f3d-45b8-46a6-800d-501a4c421332 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1095 microversion: 2.1 time: 0.076004 Sep 27 04:34:09.776132 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 508/1004] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:09 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1095 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:10.801331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:10.801987 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:10.808713 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:10.809363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:10.810029 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:10.810985 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:10.811682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:10.812146 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:10.879051 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:10.888636 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-eea87b1d-f1cd-42ef-a1df-a8130787c53f tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1201 microversion: 2.1 time: 0.093132 Sep 27 04:34:10.889205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 497/1005] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:10 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1201 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:34:11.912480 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:11.913135 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:11.920288 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:11.920772 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:11.921208 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:11.921954 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:11.922408 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:11.922829 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:12.147944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:12.157250 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5b3424da-b048-4180-b672-10b472a45386 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.248841 Sep 27 04:34:12.157848 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 509/1006] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:11 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:12.343069 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1cee166-21d9-416d-86ce-c46b3155c551", "tag": "9f31f6d4-7a24-48fa-b2fb-0449f62084ff"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:34:12.352880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:12.353454 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:12.353994 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:12.365404 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] Creating event network-changed:9f31f6d4-7a24-48fa-b2fb-0449f62084ff for instance a1cee166-21d9-416d-86ce-c46b3155c551 on hv-cinder-85918 Sep 27 04:34:12.370357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-9a6440a3-812d-4b8a-b7b6-737bed8ac6e9 req-d77d507f-3799-4343-8511-e385bfb4e077 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031649 Sep 27 04:34:12.370977 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 498/1007] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:34:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:34:13.181414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:13.182046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:13.189019 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:13.189502 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:13.189982 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:13.190666 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:13.191116 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:13.191569 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:13.392022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:13.399693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-cadcacc1-8d9d-4d1e-a0d3-eea4d8f67e79 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.222169 Sep 27 04:34:13.400256 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 510/1008] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:13 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:14.423497 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:14.424226 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:14.430563 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:14.431059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:14.431503 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:14.432190 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:14.432649 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:14.433079 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:14.563968 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:14.571611 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-fac5b481-9c89-480b-9fa4-acddc39ea2fb tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.152794 Sep 27 04:34:14.572181 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 499/1009] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:14 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:15.595522 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:15.596156 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:15.602542 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:15.603026 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:15.603465 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:15.604151 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:15.604609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:15.605054 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:15.731101 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:15.739114 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-247a862e-da9c-404d-9997-00b4f7487211 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.147480 Sep 27 04:34:15.739674 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 511/1010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:15 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:34:16.762174 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:16.763119 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:16.771123 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:16.771621 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:16.772062 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:16.773024 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:16.773890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:16.774685 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:16.909593 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:16.917128 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-443e9c19-9199-426d-9b77-15c5dbba2395 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.159290 Sep 27 04:34:16.917682 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 500/1011] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:16 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:17.939704 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:17.940343 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:17.946393 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:17.946877 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:17.947316 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:17.947999 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:17.948459 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:17.949417 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:18.078717 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:18.086356 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-60268564-b00a-445f-9b11-a23122cc67a0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.150511 Sep 27 04:34:18.086923 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 512/1012] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:17 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:19.109105 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:19.109667 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:19.116009 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:19.116508 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:19.116953 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:19.117648 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:19.118139 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:19.118571 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:19.248644 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:19.256363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-7540d71d-0010-4d10-97ee-0e1ca6ac0000 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.151797 Sep 27 04:34:19.257155 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 501/1013] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:19 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:20.280675 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:20.281310 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:20.283705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell0:SELECT=2 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:20.289261 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:20.289800 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:20.290271 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:20.290967 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:20.291426 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:20.291881 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:20.421782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:20.429720 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3c29c445-0879-4cb5-8e48-92fb9a7efe3b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.153442 Sep 27 04:34:20.430299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 513/1014] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:20 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:21.451881 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:21.452504 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:21.454460 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=2 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:21.455171 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell0:SELECT=6 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:34:21.461568 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:21.462059 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:21.462502 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:21.463189 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:21.463648 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:21.464077 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:21.599572 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:21.607652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1bca40fa-ada1-4d39-b8e7-a220b99faf5c tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.159650 Sep 27 04:34:21.608207 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 502/1015] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:21 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:22.630638 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:22.631226 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:22.638456 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:22.638945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:22.639388 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:22.640081 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:22.640543 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:22.640977 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:22.771648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:22.779242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0663afe0-ca53-463e-b489-5d0b6bb4526a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.152580 Sep 27 04:34:22.779798 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 514/1016] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:22 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:23.802913 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:23.803631 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:23.810038 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:23.810525 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:23.810962 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:23.811652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:23.812105 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:23.812535 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:23.831878 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1cee166-21d9-416d-86ce-c46b3155c551", "name": "network-vif-plugged", "status": "completed", "tag": "9f31f6d4-7a24-48fa-b2fb-0449f62084ff"}]} {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:34:23.843523 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:23.856161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:23.856161 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:23.866901 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.compute.server_external_events [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] Creating event network-vif-plugged:9f31f6d4-7a24-48fa-b2fb-0449f62084ff for instance a1cee166-21d9-416d-86ce-c46b3155c551 on hv-cinder-85918 Sep 27 04:34:23.872046 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [req-3d728426-6f76-48b9-a125-754d9ff50532 req-3561982a-f8b1-4aa2-ae16-607ea1c6f32a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044433 Sep 27 04:34:23.872885 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 515/1017] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:34:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:34:23.948846 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:23.959306 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-e62ae0c7-0f7e-4dd4-b0fb-596b202f299d tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.160124 Sep 27 04:34:23.960134 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 503/1018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:23 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:34:24.986648 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:24.987341 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:24.993706 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:24.994264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:24.994730 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:24.995946 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:24.996434 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:24.996887 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:25.126983 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:25.136163 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-0c19ebf6-f1e4-4ff9-a916-815f59549817 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1267 microversion: 2.1 time: 0.153485 Sep 27 04:34:25.136741 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 516/1019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:24 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:26.162365 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:26.163236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:26.170875 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:26.171370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:26.171811 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:26.172661 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:26.173137 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:26.173564 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:26.305329 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:26.313797 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-316afdb7-92c8-47a6-83a1-b392ad5ba8d0 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1454 microversion: 2.1 time: 0.157380 Sep 27 04:34:26.314377 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 504/1020] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:26 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1454 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 04:34:26.336204 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:34:26.336839 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:34:26.343424 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:26.343909 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:26.344350 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:26.345036 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:26.345488 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:26.345975 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:26.476655 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:34:26.488624 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f0c6bd69-9eaa-413f-afe1-bbc05a750ac8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1454 microversion: 2.1 time: 0.156212 Sep 27 04:34:26.489222 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 517/1021] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:34:26 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1454 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:34:28.066302 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1cee166-21d9-416d-86ce-c46b3155c551", "tag": "9f31f6d4-7a24-48fa-b2fb-0449f62084ff"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:34:28.077381 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:34:28.077899 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:34:28.078331 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:34:28.089230 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] Creating event network-changed:9f31f6d4-7a24-48fa-b2fb-0449f62084ff for instance a1cee166-21d9-416d-86ce-c46b3155c551 on hv-cinder-85918 Sep 27 04:34:28.094457 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-80d0750e-6170-42e7-86b0-a72dbd4e17b1 req-b2820888-5290-456f-83cd-9d0c83ffedd3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031786 Sep 27 04:34:28.095128 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 505/1022] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:34:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:35:57.459782 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_api:SELECT=22 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:35:57.461014 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG dbcounter [-] [120423] Writing DB stats nova_cell1:SELECT=50 {{(pid=120423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:35:57.462625 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:35:57.463052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 04:35:57.463052 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Tue Sep 27 04:35:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 (ip 10.222.0.26) !!! Sep 27 04:35:57.463792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG dbcounter [-] [120424] Writing DB stats nova_cell1:SELECT=48 {{(pid=120424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 04:35:57.469978 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:35:57.470591 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:35:57.471383 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1cee166-21d9-416d-86ce-c46b3155c551", "tag": "9f31f6d4-7a24-48fa-b2fb-0449f62084ff"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:35:57.480883 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:57.481479 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:57.482008 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:57.482781 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:57.483307 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:57.483744 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:57.484862 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:57.485370 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:57.485827 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:57.498214 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] Creating event network-changed:9f31f6d4-7a24-48fa-b2fb-0449f62084ff for instance a1cee166-21d9-416d-86ce-c46b3155c551 on hv-cinder-85918 Sep 27 04:35:57.503632 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-6b4cbdd9-b452-4403-8588-9e7da2cc9388 req-94bf519e-07de-4008-a8e8-4e2a68e4b61f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043370 Sep 27 04:35:57.504444 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 506/1023] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:35:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 04:35:57.513460 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:35:57.600242 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-3cfee29c-1121-4b87-bcc9-bf4a43758b7e tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 204 len: 0 microversion: 2.1 time: 0.142330 Sep 27 04:35:57.603996 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 518/1024] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:35:57 2022] DELETE /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:35:57.610612 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:35:57.611292 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:35:57.617736 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:57.618268 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:57.618730 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:57.619425 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:57.619883 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:57.620310 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:57.875703 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:35:57.883490 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-6159ebb0-9cb1-4fba-869c-e20dd870175a tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1579 microversion: 2.1 time: 0.277345 Sep 27 04:35:57.884058 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 507/1025] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:35:57 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1579 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:35:58.911117 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:35:58.911816 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:35:58.918362 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:58.918846 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:58.919282 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:58.919963 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:58.920415 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:58.920845 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:59.168286 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:35:59.175851 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-f02e141f-8224-4463-b926-43bda3ab6701 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1579 microversion: 2.1 time: 0.269310 Sep 27 04:35:59.176407 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 519/1026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:35:58 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1579 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:35:59.957509 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1cee166-21d9-416d-86ce-c46b3155c551", "name": "network-vif-deleted", "tag": "9f31f6d4-7a24-48fa-b2fb-0449f62084ff"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:35:59.968272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:35:59.968767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:35:59.969205 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:35:59.981017 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] Creating event network-vif-deleted:9f31f6d4-7a24-48fa-b2fb-0449f62084ff for instance a1cee166-21d9-416d-86ce-c46b3155c551 on hv-cinder-85918 Sep 27 04:35:59.986813 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-b11a91d1-9bd8-401c-9452-2ac674810185 req-dd662561-f5de-4d37-95d3-2d5d7b916ef2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033037 Sep 27 04:35:59.987418 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 508/1027] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:35:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:36:00.200139 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:00.200697 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:00.207069 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:00.207554 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:00.207987 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:00.208668 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:00.209118 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:00.209567 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:00.283264 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:36:00.295031 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d8209f1a-1d7a-4ace-b76d-b181548bd55b tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 200 len: 1226 microversion: 2.1 time: 0.098871 Sep 27 04:36:00.295609 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 520/1028] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:00 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:36:01.324490 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:01.325193 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: a1cee166-21d9-416d-86ce-c46b3155c551] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:01.331227 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:01.331715 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:01.332151 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:01.332830 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:01.333287 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:01.333711 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:01.347185 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] HTTP exception thrown: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Request method failure captured: Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 HTTP/1.1 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnwsQBwYHnQ_PGyzlt_S6KBiJbFTHyTPilJAkq2AL5BQ77owISesAEMfIqIUR3FWdXUwaEDuLEcrqIBmDSEKYiZ3xSNW60INcigHq0jDH-I5XcgoLn1v1Ij32MUqYlASpSbDpmFwsA2VXFwj-ibq_oL1l2G9DWZUjgTpmX8kc1Yf6w4 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:36:01.348770 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_53d86b6e8a8343fca03ca6d38200f23f", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 44a710d801334b20b9f433eaad61cbc6 Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. Sep 27 04:36:01.350545 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': 'a1cee166-21d9-416d-86ce-c46b3155c551'} Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:36:01.351550 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. Sep 27 04:36:01.353263 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:01.354767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Returning 404 to user: Instance a1cee166-21d9-416d-86ce-c46b3155c551 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:36:01.354767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-27e602b3-33bc-4027-abc8-d6ae6d352104 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551" status: 404 len: 111 microversion: 2.1 time: 0.030907 Sep 27 04:36:01.354767 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 509/1029] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:01 2022] GET /compute/v2.1/servers/a1cee166-21d9-416d-86ce-c46b3155c551 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:36:04.409679 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56108588-5052-4cbc-b248-29229aef7aec", "tag": "f9768712-2949-4085-b485-9a6386391423"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:36:04.416612 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:04.418351 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:04.424693 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:04.425191 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:04.425632 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:04.426414 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:04.426907 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:04.427807 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:04.428385 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:04.428871 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:04.429681 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:04.441363 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] Creating event network-changed:f9768712-2949-4085-b485-9a6386391423 for instance 56108588-5052-4cbc-b248-29229aef7aec on hv-cinder-85918 Sep 27 04:36:04.446430 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-f6a2e3af-2635-4127-85f0-6a4ad97582ed req-5f0aba3a-06e6-4591-9623-8297a258606b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040345 Sep 27 04:36:04.447272 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 510/1030] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:36:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:36:04.454694 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Going to try to terminate instance {{(pid=120423) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 04:36:04.547945 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-d193be83-673d-4d3d-9687-5836af28d475 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "DELETE /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 204 len: 0 microversion: 2.1 time: 0.144318 Sep 27 04:36:04.548532 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 521/1031] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:36:04 2022] DELETE /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 04:36:04.557352 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:04.558888 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:04.565290 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:04.565886 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:04.566567 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:04.567334 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:04.567869 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:04.568633 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:04.709589 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.policy [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120424) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:36:04.717285 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-1b12aa05-25e4-4356-bca3-b2cb21d4b9f8 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1579 microversion: 2.1 time: 0.164122 Sep 27 04:36:04.717880 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 511/1032] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:04 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1579 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:36:05.740974 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:05.741703 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:05.748532 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:05.749022 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:05.749458 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:05.750436 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:05.750919 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:05.751346 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:05.954682 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:36:05.962319 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-e655a51c-0db8-4e29-a11f-9daeb0090c98 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1579 microversion: 2.1 time: 0.225630 Sep 27 04:36:05.962884 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 522/1033] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:05 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1579 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:36:06.711890 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56108588-5052-4cbc-b248-29229aef7aec", "name": "network-vif-deleted", "tag": "f9768712-2949-4085-b485-9a6386391423"}]} {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 04:36:06.722043 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:06.722530 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:06.722966 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:06.734652 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.compute.server_external_events [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] Creating event network-vif-deleted:f9768712-2949-4085-b485-9a6386391423 for instance 56108588-5052-4cbc-b248-29229aef7aec on hv-cinder-85918 Sep 27 04:36:06.740284 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [req-1cd13937-152c-4b36-99ae-56f83fdd7413 req-bdbb199d-c4b8-4866-8961-05c83b44a2ed service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031856 Sep 27 04:36:06.741114 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 512/1034] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 04:36:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 04:36:06.986827 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:06.987425 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:06.994805 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:06.996094 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:06.996566 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:06.997444 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:06.997944 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:06.998375 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:07.035715 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:07.036334 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:07.036771 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:07.037359 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.objects.instance [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lazy-loading 'fault' on Instance uuid 56108588-5052-4cbc-b248-29229aef7aec {{(pid=120423) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 04:36:07.081689 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.policy [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44a710d801334b20b9f433eaad61cbc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53d86b6e8a8343fca03ca6d38200f23f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120423) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 04:36:07.090490 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-9fe22ff3-2524-4917-b086-0d403f5e04df tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 200 len: 1231 microversion: 2.1 time: 0.107537 Sep 27 04:36:07.091065 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 523/1035] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:06 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 1231 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 04:36:08.117036 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:08.117733 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 56108588-5052-4cbc-b248-29229aef7aec] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:08.123873 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.124354 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.124792 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.125753 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.126208 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.126635 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.140122 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] HTTP exception thrown: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Request method failure captured: Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec HTTP/1.1 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnwsQBwYHnQ_PGyzlt_S6KBiJbFTHyTPilJAkq2AL5BQ77owISesAEMfIqIUR3FWdXUwaEDuLEcrqIBmDSEKYiZ3xSNW60INcigHq0jDH-I5XcgoLn1v1Ij32MUqYlASpSbDpmFwsA2VXFwj-ibq_oL1l2G9DWZUjgTpmX8kc1Yf6w4 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:36:08.141707 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_53d86b6e8a8343fca03ca6d38200f23f", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 44a710d801334b20b9f433eaad61cbc6 Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. Sep 27 04:36:08.143250 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '56108588-5052-4cbc-b248-29229aef7aec'} Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:36:08.144109 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. Sep 27 04:36:08.145820 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.147357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Returning 404 to user: Instance 56108588-5052-4cbc-b248-29229aef7aec could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:36:08.147357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-8db7f304-734e-48ca-b875-e03fbc16d4b1 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec" status: 404 len: 111 microversion: 2.1 time: 0.030817 Sep 27 04:36:08.147357 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 513/1036] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:08 2022] GET /compute/v2.1/servers/56108588-5052-4cbc-b248-29229aef7aec => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:36:08.574784 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:08.575295 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.compute.api [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120423) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:08.581833 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.582279 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.582721 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.583410 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.583868 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.584299 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG oslo_concurrency.lockutils [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120423) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.597203 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.wsgi [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] HTTP exception thrown: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Request method failure captured: Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: request: DELETE /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 HTTP/1.1 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept: application/json Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Accept-Encoding: identity Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Connection: close Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Length: 0 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Content-Type: application/json Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Host: 10.222.0.26 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: Openstack-System-Scope: None Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: User-Agent: python-urllib3/1.26.12 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Auth-Token: gAAAAABjMnwsQBwYHnQ_PGyzlt_S6KBiJbFTHyTPilJAkq2AL5BQ77owISesAEMfIqIUR3FWdXUwaEDuLEcrqIBmDSEKYiZ3xSNW60INcigHq0jDH-I5XcgoLn1v1Ij32MUqYlASpSbDpmFwsA2VXFwj-ibq_oL1l2G9DWZUjgTpmX8kc1Yf6w4 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Id: None Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Domain-Name: None Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Identity-Status: Confirmed Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Is-Admin-Project: True Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Id: default Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Domain-Name: Default Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Project-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Role: reader,member Sep 27 04:36:08.598875 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Roles: reader,member Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_53d86b6e8a8343fca03ca6d38200f23f", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-Tenant-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Id: default Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Domain-Name: Default Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Id: 44a710d801334b20b9f433eaad61cbc6 Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: X-User-Name: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: method: > Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: exception: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.600592 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: action_args: {'id': '298d0196-0c87-4bfd-a862-be6cbe3a28f5'} Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:36:08.601705 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.603600 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.605190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Returning 404 to user: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. {{(pid=120423) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:36:08.605190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-5b799501-92b3-4ab6-b7b8-cf5a1af2bece tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "DELETE /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 404 len: 111 microversion: 2.1 time: 0.030148 Sep 27 04:36:08.605190 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 524/1037] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 04:36:08 2022] DELETE /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 04:36:08.608793 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '>' {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:08.609435 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.compute.api [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] [instance: 298d0196-0c87-4bfd-a862-be6cbe3a28f5] Fetching instance by UUID {{(pid=120424) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 04:36:08.615236 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.615725 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.616167 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.616844 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Acquiring lock "68475614-ed24-4318-8ce9-852b74c0d16a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 04:36:08.617300 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 04:36:08.617724 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG oslo_concurrency.lockutils [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Lock "68475614-ed24-4318-8ce9-852b74c0d16a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120424) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 04:36:08.631690 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.wsgi [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] HTTP exception thrown: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Request method failure captured: Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: request: GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 HTTP/1.1 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept: application/json Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Accept-Encoding: identity Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Connection: close Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Length: 0 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Content-Type: application/json Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Host: 10.222.0.26 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: Openstack-System-Scope: None Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: User-Agent: python-urllib3/1.26.12 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Auth-Token: gAAAAABjMnwsQBwYHnQ_PGyzlt_S6KBiJbFTHyTPilJAkq2AL5BQ77owISesAEMfIqIUR3FWdXUwaEDuLEcrqIBmDSEKYiZ3xSNW60INcigHq0jDH-I5XcgoLn1v1Ij32MUqYlASpSbDpmFwsA2VXFwj-ibq_oL1l2G9DWZUjgTpmX8kc1Yf6w4 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Id: None Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Domain-Name: None Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Identity-Status: Confirmed Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Is-Admin-Project: True Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Id: default Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Domain-Name: Default Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Project-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Role: reader,member Sep 27 04:36:08.633173 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Roles: reader,member Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:8080/v1/AUTH_53d86b6e8a8343fca03ca6d38200f23f", "adminURL": "http://10.222.0.26:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/53d86b6e8a8343fca03ca6d38200f23f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Id: 53d86b6e8a8343fca03ca6d38200f23f Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-Tenant-Name: tempest-TestVolumeBootPattern-57324624 Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Id: default Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Domain-Name: Default Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Id: 44a710d801334b20b9f433eaad61cbc6 Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: X-User-Name: tempest-TestVolumeBootPattern-57324624-project Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: method: > Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: exception: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.635050 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: action_args: {'id': '298d0196-0c87-4bfd-a862-be6cbe3a28f5'} Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: {{(pid=120424) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 04:36:08.636539 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. Sep 27 04:36:08.638135 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: ERROR nova.api.openstack.wsgi Sep 27 04:36:08.639653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: DEBUG nova.api.openstack.wsgi [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Returning 404 to user: Instance 298d0196-0c87-4bfd-a862-be6cbe3a28f5 could not be found. {{(pid=120424) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 04:36:08.639653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: INFO nova.api.openstack.requestlog [None req-908a7333-fac4-4d2a-857b-e1989f98ec11 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5" status: 404 len: 111 microversion: 2.1 time: 0.029614 Sep 27 04:36:08.639653 dv-cinder-859186-3-iscsi devstack@n-api.service[120424]: [pid: 120424|app: 0|req: 514/1038] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 04:36:08 2022] GET /compute/v2.1/servers/298d0196-0c87-4bfd-a862-be6cbe3a28f5 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 04:36:09.925239 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: DEBUG nova.api.openstack.wsgi [None req-2f91492c-5624-42a3-b956-16486b422523 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] Calling method '.version_select at 0x7fce70be9430>' {{(pid=120423) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 04:36:09.948580 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: INFO nova.api.openstack.requestlog [None req-2f91492c-5624-42a3-b956-16486b422523 tempest-TestVolumeBootPattern-57324624 tempest-TestVolumeBootPattern-57324624-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1258945985" status: 202 len: 0 microversion: 2.1 time: 0.026441 Sep 27 04:36:09.949893 dv-cinder-859186-3-iscsi devstack@n-api.service[120423]: [pid: 120423|app: 0|req: 525/1039] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 04:36:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1258945985 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)